2025-09-01 Mon 02:03:54:041.362 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011171994240, Pid 113882]: vhost interface name : vhost0 2025-09-01 Mon 02:03:54:041.647 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011171994240, Pid 113882]: vhost IP Address : 10.20.0.14/24 2025-09-01 Mon 02:03:54:041.670 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011171994240, Pid 113882]: vhost gateway : 0.0.0.0 2025-09-01 Mon 02:03:54:041.686 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011171994240, Pid 113882]: Ethernet ports : ens6 2025-09-01 Mon 02:03:54:041.705 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011171994240, Pid 113882]: Loopback IP : 0.0.0.0 2025-09-01 Mon 02:03:54:041.729 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011171994240, Pid 113882]: Xmpp Servers : 10.20.0.14:5269 2025-09-01 Mon 02:03:54:041.755 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011171994240, Pid 113882]: Xmpp Authentication : 0 2025-09-01 Mon 02:03:54:041.777 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011171994240, Pid 113882]: Cluster-Name : 2025-09-01 Mon 02:03:54:041.800 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011171994240, Pid 113882]: DNS Servers : 10.20.0.14:53 2025-09-01 Mon 02:03:54:041.822 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011171994240, Pid 113882]: DNS client port : 8098 2025-09-01 Mon 02:03:54:041.845 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011171994240, Pid 113882]: DNS timeout : 3000 2025-09-01 Mon 02:03:54:041.867 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011171994240, Pid 113882]: DNS max retries : 2 2025-09-01 Mon 02:03:54:041.888 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011171994240, Pid 113882]: Xmpp Dns Authentication : 0 2025-09-01 Mon 02:03:54:041.908 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011171994240, Pid 113882]: TSN Servers : [] 2025-09-01 Mon 02:03:54:041.924 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011171994240, Pid 113882]: COLLECTOR Servers : 10.0.0.56:8086 2025-09-01 Mon 02:03:54:041.939 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011171994240, Pid 113882]: Tunnel-Type : MPLSoGRE 2025-09-01 Mon 02:03:54:041.953 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011171994240, Pid 113882]: Metadata-Proxy Shared Secret: contrail 2025-09-01 Mon 02:03:54:041.968 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011171994240, Pid 113882]: Metadata-Proxy Port : 8097 2025-09-01 Mon 02:03:54:041.982 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011171994240, Pid 113882]: Metadata-Proxy SSL Flag : 0 2025-09-01 Mon 02:03:54:041.997 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011171994240, Pid 113882]: Linklocal Max System Flows : 16384 2025-09-01 Mon 02:03:54:042.011 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011171994240, Pid 113882]: Linklocal Max Vm Flows : 16384 2025-09-01 Mon 02:03:54:042.026 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011171994240, Pid 113882]: Flow cache timeout : 0 2025-09-01 Mon 02:03:54:042.040 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011171994240, Pid 113882]: Stale Interface cleanup timeout : 60 2025-09-01 Mon 02:03:54:042.055 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011171994240, Pid 113882]: Flow thread count : 1 2025-09-01 Mon 02:03:54:042.070 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011171994240, Pid 113882]: Flow latency limit : 0 2025-09-01 Mon 02:03:54:042.084 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011171994240, Pid 113882]: Flow index-mgr sm log count : 0 2025-09-01 Mon 02:03:54:042.099 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011171994240, Pid 113882]: Flow add-tokens : 50 2025-09-01 Mon 02:03:54:042.127 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011171994240, Pid 113882]: Flow ksync-tokens : 25 2025-09-01 Mon 02:03:54:042.142 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011171994240, Pid 113882]: Flow del-tokens : 16 2025-09-01 Mon 02:03:54:042.157 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011171994240, Pid 113882]: Flow update-tokens : 16 2025-09-01 Mon 02:03:54:042.171 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011171994240, Pid 113882]: Pin flow netlink task to CPU: 2025-09-01 Mon 02:03:54:042.186 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011171994240, Pid 113882]: Maximum sessions : 100 2025-09-01 Mon 02:03:54:042.221 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011171994240, Pid 113882]: Maximum session aggregates : 8 2025-09-01 Mon 02:03:54:042.237 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011171994240, Pid 113882]: Maximum session endpoints : 5 2025-09-01 Mon 02:03:54:042.251 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011171994240, Pid 113882]: Fabric SNAT hash table size : 4096 2025-09-01 Mon 02:03:54:042.266 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011171994240, Pid 113882]: Flow excluding Router ID in hash :0 2025-09-01 Mon 02:03:54:042.284 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011171994240, Pid 113882]: Agent Mode : Vrouter 2025-09-01 Mon 02:03:54:042.298 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011171994240, Pid 113882]: Gateway Mode : None 2025-09-01 Mon 02:03:54:042.313 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011171994240, Pid 113882]: DHCP Relay Mode : 0 2025-09-01 Mon 02:03:54:042.328 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011171994240, Pid 113882]: Service instance netns cmd : /usr/local/bin/opencontrail-vrouter-netns 2025-09-01 Mon 02:03:54:042.345 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011171994240, Pid 113882]: Service instance docker cmd : /usr/local/bin/opencontrail-vrouter-docker 2025-09-01 Mon 02:03:54:042.360 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011171994240, Pid 113882]: Service instance workers : 0 2025-09-01 Mon 02:03:54:042.375 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011171994240, Pid 113882]: Service instance timeout : 0 2025-09-01 Mon 02:03:54:042.389 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011171994240, Pid 113882]: Service instance lb ssl : 2025-09-01 Mon 02:03:54:042.404 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011171994240, Pid 113882]: Service instance lbaas auth : 2025-09-01 Mon 02:03:54:042.418 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011171994240, Pid 113882]: Bgp as a service port range : 50000-50512 2025-09-01 Mon 02:03:54:042.433 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011171994240, Pid 113882]: Services queue limit : 1024 2025-09-01 Mon 02:03:54:042.447 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011171994240, Pid 113882]: BGPAAS max shared sessions for service port : 4 2025-09-01 Mon 02:03:54:042.462 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011171994240, Pid 113882]: Sandesh Key file : /etc/contrail/ssl/private/server-privkey.pem 2025-09-01 Mon 02:03:54:042.476 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011171994240, Pid 113882]: Sandesh Cert file : /etc/contrail/ssl/certs/server.pem 2025-09-01 Mon 02:03:54:042.491 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011171994240, Pid 113882]: Sandesh CA Cert : /etc/contrail/ssl/certs/ca-cert.pem 2025-09-01 Mon 02:03:54:042.512 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011171994240, Pid 113882]: Sandesh SSL Enable : 0 2025-09-01 Mon 02:03:54:042.545 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011171994240, Pid 113882]: Introspect SSL Enable : 0 2025-09-01 Mon 02:03:54:042.566 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011171994240, Pid 113882]: Hypervisor mode : kvm 2025-09-01 Mon 02:03:54:042.812 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011171994240, Pid 113882]: Ethernet Port Encap Type : 2025-09-01 Mon 02:03:54:042.836 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011171994240, Pid 113882]: Platform mode : Vrouter on host linux kernel 2025-09-01 Mon 02:03:54:043.388 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: SANDESH: Logging: DISABLED -> ENABLED 2025-09-01 Mon 02:03:54:043.443 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: SANDESH: Logging: LEVEL: [ INVALID ] -> [ SYS_DEBUG ] log4level: [ NOTSET ] -> [ DEBUG ] 2025-09-01 Mon 02:03:54:043.466 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011171994240, Pid 113882]: VRouter [SYS_INFO]: VersionInfoLog: {"build-info":[{"build-version":"v0+nightly","build-time":"2025-09-01 00:13:09.128962","build-user":"root","build-hostname":"jenkins-build-prepare-sandbox-10717-1.novalocal","build-id":"unknown","build-number":"unknown"}]} src/contrail-common/base/misc_utils.cc 41 2025-09-01 Mon 02:03:54:062.232 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Updating dependency tacker rules for virtual-network 2025-09-01 Mon 02:03:54:062.334 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Adding ReactorMap virtual-network : virtual-network-routing-instance -> self 2025-09-01 Mon 02:03:54:062.358 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Adding ReactorMap routing-instance : self -> virtual-network-routing-instance 2025-09-01 Mon 02:03:54:062.376 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Adding ReactorMap routing-instance : virtual-network-routing-instance -> nil 2025-09-01 Mon 02:03:54:062.395 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Updating dependency tacker rules for virtual-network 2025-09-01 Mon 02:03:54:062.411 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Adding ReactorMap virtual-network : virtual-network-access-control-list -> self 2025-09-01 Mon 02:03:54:062.428 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Adding ReactorMap access-control-list : self -> virtual-network-access-control-list 2025-09-01 Mon 02:03:54:062.444 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Adding ReactorMap access-control-list : virtual-network-access-control-list -> nil 2025-09-01 Mon 02:03:54:062.462 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Updating dependency tacker rules for virtual-network 2025-09-01 Mon 02:03:54:062.478 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Adding ReactorMap virtual-network : virtual-network-network-ipam -> self 2025-09-01 Mon 02:03:54:062.494 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Adding ReactorMap virtual-network-network-ipam : self -> virtual-network-network-ipam 2025-09-01 Mon 02:03:54:062.512 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Adding ReactorMap virtual-network-network-ipam : virtual-network-network-ipam -> virtual-network-network-ipam 2025-09-01 Mon 02:03:54:062.528 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Adding ReactorMap network-ipam : virtual-network-network-ipam -> nil 2025-09-01 Mon 02:03:54:062.546 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Updating dependency tacker rules for virtual-network 2025-09-01 Mon 02:03:54:062.577 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Adding ReactorMap virtual-network : virtual-network-qos-config -> self 2025-09-01 Mon 02:03:54:062.596 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Adding ReactorMap qos-config : self -> virtual-network-qos-config 2025-09-01 Mon 02:03:54:062.612 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Adding ReactorMap qos-config : virtual-network-qos-config -> nil 2025-09-01 Mon 02:03:54:062.629 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Updating dependency tacker rules for virtual-network 2025-09-01 Mon 02:03:54:062.644 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Adding ReactorMap virtual-network : virtual-network-security-logging-object -> self 2025-09-01 Mon 02:03:54:062.661 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Adding ReactorMap security-logging-object : self -> virtual-network-security-logging-object 2025-09-01 Mon 02:03:54:062.677 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Adding ReactorMap security-logging-object : virtual-network-security-logging-object -> nil 2025-09-01 Mon 02:03:54:062.694 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Updating dependency tacker rules for virtual-network 2025-09-01 Mon 02:03:54:062.709 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Adding ReactorMap virtual-network : logical-router-virtual-network -> self 2025-09-01 Mon 02:03:54:062.727 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Adding ReactorMap logical-router-virtual-network : self -> logical-router-virtual-network 2025-09-01 Mon 02:03:54:062.743 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Adding ReactorMap logical-router-virtual-network : logical-router-virtual-network -> logical-router-virtual-network 2025-09-01 Mon 02:03:54:062.760 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Adding ReactorMap logical-router : self -> logical-router-virtual-network 2025-09-01 Mon 02:03:54:062.776 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Adding ReactorMap logical-router : logical-router-virtual-network -> nil 2025-09-01 Mon 02:03:54:062.794 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Updating dependency tacker rules for virtual-network 2025-09-01 Mon 02:03:54:062.809 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Adding ReactorMap virtual-network : virtual-network-multicast-policy -> self 2025-09-01 Mon 02:03:54:062.826 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Adding ReactorMap multicast-policy : self -> virtual-network-multicast-policy 2025-09-01 Mon 02:03:54:062.842 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Adding ReactorMap multicast-policy : virtual-network-multicast-policy -> nil 2025-09-01 Mon 02:03:54:062.861 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Updating dependency tacker rules for routing-instance 2025-09-01 Mon 02:03:54:062.899 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Adding ReactorMap routing-instance : virtual-network-routing-instance -> self 2025-09-01 Mon 02:03:54:062.914 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Adding ReactorMap virtual-network : self -> virtual-network-routing-instance 2025-09-01 Mon 02:03:54:062.931 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Adding ReactorMap virtual-network : virtual-network-provider-network -> virtual-network-routing-instance 2025-09-01 Mon 02:03:54:062.954 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Adding ReactorMap virtual-network : self -> virtual-network-provider-network 2025-09-01 Mon 02:03:54:062.970 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Adding ReactorMap virtual-network : virtual-network-provider-network -> nil 2025-09-01 Mon 02:03:54:062.990 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Updating dependency tacker rules for security-group 2025-09-01 Mon 02:03:54:063.005 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Adding ReactorMap security-group : security-group-access-control-list -> self 2025-09-01 Mon 02:03:54:063.021 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Adding ReactorMap access-control-list : self -> security-group-access-control-list 2025-09-01 Mon 02:03:54:063.037 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Adding ReactorMap access-control-list : security-group-access-control-list -> nil 2025-09-01 Mon 02:03:54:063.057 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Updating dependency tacker rules for tag 2025-09-01 Mon 02:03:54:063.073 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Adding ReactorMap tag : application-policy-set-tag -> self 2025-09-01 Mon 02:03:54:063.090 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Adding ReactorMap application-policy-set : self -> application-policy-set-tag 2025-09-01 Mon 02:03:54:063.106 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Adding ReactorMap application-policy-set : policy-management-application-policy-set -> application-policy-set-tag 2025-09-01 Mon 02:03:54:063.124 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Adding ReactorMap policy-management : policy-management-application-policy-set -> nil 2025-09-01 Mon 02:03:54:063.142 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Updating dependency tacker rules for virtual-machine-interface 2025-09-01 Mon 02:03:54:063.157 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Adding ReactorMap virtual-machine-interface : virtual-machine-interface-virtual-network -> self 2025-09-01 Mon 02:03:54:063.173 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Adding ReactorMap virtual-network : self -> virtual-machine-interface-virtual-network 2025-09-01 Mon 02:03:54:063.217 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Adding ReactorMap virtual-network : virtual-network-network-ipam -> virtual-machine-interface-virtual-network 2025-09-01 Mon 02:03:54:063.234 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Adding ReactorMap virtual-network-network-ipam : self -> virtual-network-network-ipam 2025-09-01 Mon 02:03:54:063.250 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Adding ReactorMap virtual-network-network-ipam : virtual-network-network-ipam -> nil 2025-09-01 Mon 02:03:54:063.267 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Updating dependency tacker rules for virtual-machine-interface 2025-09-01 Mon 02:03:54:063.282 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Adding ReactorMap virtual-machine-interface : virtual-machine-interface-virtual-machine -> self 2025-09-01 Mon 02:03:54:063.298 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Adding ReactorMap virtual-machine : self -> virtual-machine-interface-virtual-machine 2025-09-01 Mon 02:03:54:063.322 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Adding ReactorMap virtual-machine : virtual-machine-interface-virtual-machine -> nil 2025-09-01 Mon 02:03:54:063.339 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Updating dependency tacker rules for virtual-machine-interface 2025-09-01 Mon 02:03:54:063.354 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Adding ReactorMap virtual-machine-interface : virtual-machine-interface-sub-interface -> self 2025-09-01 Mon 02:03:54:063.370 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Adding ReactorMap virtual-machine-interface : self -> virtual-machine-interface-sub-interface 2025-09-01 Mon 02:03:54:063.385 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Adding ReactorMap virtual-machine-interface : virtual-machine-interface-sub-interface -> nil 2025-09-01 Mon 02:03:54:063.402 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Updating dependency tacker rules for virtual-machine-interface 2025-09-01 Mon 02:03:54:063.417 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Adding ReactorMap virtual-machine-interface : virtual-machine-interface-virtual-network -> self 2025-09-01 Mon 02:03:54:063.432 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Adding ReactorMap virtual-network : self -> virtual-machine-interface-virtual-network 2025-09-01 Mon 02:03:54:063.448 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Adding ReactorMap virtual-network : virtual-network-provider-network -> virtual-machine-interface-virtual-network 2025-09-01 Mon 02:03:54:063.463 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Adding ReactorMap virtual-network : self -> virtual-network-provider-network 2025-09-01 Mon 02:03:54:063.478 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Adding ReactorMap virtual-network : virtual-network-provider-network -> nil 2025-09-01 Mon 02:03:54:063.496 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Updating dependency tacker rules for virtual-machine-interface 2025-09-01 Mon 02:03:54:063.511 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Adding ReactorMap virtual-machine-interface : virtual-machine-interface-routing-instance -> self 2025-09-01 Mon 02:03:54:063.529 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Adding ReactorMap virtual-machine-interface-routing-instance : self -> virtual-machine-interface-routing-instance 2025-09-01 Mon 02:03:54:063.545 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Adding ReactorMap virtual-machine-interface-routing-instance : virtual-machine-interface-routing-instance -> virtual-machine-interface-routing-instance 2025-09-01 Mon 02:03:54:063.561 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Adding ReactorMap routing-instance : self -> virtual-machine-interface-routing-instance 2025-09-01 Mon 02:03:54:063.577 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Adding ReactorMap routing-instance : virtual-machine-interface-routing-instance -> nil 2025-09-01 Mon 02:03:54:063.594 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Updating dependency tacker rules for virtual-machine-interface 2025-09-01 Mon 02:03:54:063.608 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Adding ReactorMap virtual-machine-interface : virtual-machine-interface-security-group -> self 2025-09-01 Mon 02:03:54:063.624 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Adding ReactorMap security-group : self -> virtual-machine-interface-security-group 2025-09-01 Mon 02:03:54:063.646 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Adding ReactorMap security-group : virtual-machine-interface-security-group -> nil 2025-09-01 Mon 02:03:54:063.665 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Updating dependency tacker rules for virtual-machine-interface 2025-09-01 Mon 02:03:54:063.680 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Adding ReactorMap virtual-machine-interface : alias-ip-virtual-machine-interface -> self 2025-09-01 Mon 02:03:54:063.697 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Adding ReactorMap alias-ip : self -> alias-ip-virtual-machine-interface 2025-09-01 Mon 02:03:54:063.713 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Adding ReactorMap alias-ip : alias-ip-pool-alias-ip -> alias-ip-virtual-machine-interface 2025-09-01 Mon 02:03:54:063.731 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Adding ReactorMap alias-ip-pool : virtual-network-alias-ip-pool -> alias-ip-pool-alias-ip 2025-09-01 Mon 02:03:54:063.747 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Adding ReactorMap virtual-network : self -> virtual-network-alias-ip-pool 2025-09-01 Mon 02:03:54:063.784 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Adding ReactorMap virtual-network : virtual-network-alias-ip-pool -> nil 2025-09-01 Mon 02:03:54:063.805 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Updating dependency tacker rules for virtual-machine-interface 2025-09-01 Mon 02:03:54:063.820 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Adding ReactorMap virtual-machine-interface : floating-ip-virtual-machine-interface -> self 2025-09-01 Mon 02:03:54:063.837 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Adding ReactorMap floating-ip : self -> floating-ip-virtual-machine-interface 2025-09-01 Mon 02:03:54:063.853 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Adding ReactorMap floating-ip : floating-ip-pool-floating-ip -> floating-ip-virtual-machine-interface 2025-09-01 Mon 02:03:54:063.871 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Adding ReactorMap floating-ip-pool : virtual-network-floating-ip-pool -> floating-ip-pool-floating-ip 2025-09-01 Mon 02:03:54:063.887 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Adding ReactorMap virtual-network : self -> virtual-network-floating-ip-pool 2025-09-01 Mon 02:03:54:063.903 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Adding ReactorMap virtual-network : virtual-network-provider-network -> virtual-network-floating-ip-pool 2025-09-01 Mon 02:03:54:063.918 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Adding ReactorMap virtual-network : self -> virtual-network-provider-network 2025-09-01 Mon 02:03:54:063.933 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Adding ReactorMap virtual-network : virtual-network-provider-network -> nil 2025-09-01 Mon 02:03:54:063.952 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Updating dependency tacker rules for virtual-machine-interface 2025-09-01 Mon 02:03:54:063.967 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Adding ReactorMap virtual-machine-interface : floating-ip-virtual-machine-interface -> self 2025-09-01 Mon 02:03:54:063.982 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Adding ReactorMap floating-ip : self -> floating-ip-virtual-machine-interface 2025-09-01 Mon 02:03:54:064.010 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Adding ReactorMap floating-ip : floating-ip-pool-floating-ip -> floating-ip-virtual-machine-interface 2025-09-01 Mon 02:03:54:064.026 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Adding ReactorMap floating-ip-pool : virtual-network-floating-ip-pool -> floating-ip-pool-floating-ip 2025-09-01 Mon 02:03:54:064.041 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Adding ReactorMap virtual-network : self -> virtual-network-floating-ip-pool 2025-09-01 Mon 02:03:54:064.057 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Adding ReactorMap virtual-network : virtual-network-routing-instance -> virtual-network-floating-ip-pool 2025-09-01 Mon 02:03:54:064.072 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Adding ReactorMap routing-instance : self -> virtual-network-routing-instance 2025-09-01 Mon 02:03:54:064.088 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Adding ReactorMap routing-instance : virtual-network-routing-instance -> nil 2025-09-01 Mon 02:03:54:064.105 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Updating dependency tacker rules for virtual-machine-interface 2025-09-01 Mon 02:03:54:064.119 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Adding ReactorMap virtual-machine-interface : floating-ip-virtual-machine-interface -> self 2025-09-01 Mon 02:03:54:064.135 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Adding ReactorMap floating-ip : self -> floating-ip-virtual-machine-interface 2025-09-01 Mon 02:03:54:064.150 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Adding ReactorMap floating-ip : instance-ip-floating-ip -> floating-ip-virtual-machine-interface 2025-09-01 Mon 02:03:54:064.166 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Adding ReactorMap instance-ip : instance-ip-virtual-network -> instance-ip-floating-ip 2025-09-01 Mon 02:03:54:064.182 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Adding ReactorMap virtual-network : self -> instance-ip-virtual-network 2025-09-01 Mon 02:03:54:064.212 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Adding ReactorMap virtual-network : instance-ip-virtual-network -> nil 2025-09-01 Mon 02:03:54:064.229 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Updating dependency tacker rules for virtual-machine-interface 2025-09-01 Mon 02:03:54:064.244 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Adding ReactorMap virtual-machine-interface : instance-ip-virtual-machine-interface -> self 2025-09-01 Mon 02:03:54:064.259 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Adding ReactorMap instance-ip : self -> instance-ip-virtual-machine-interface 2025-09-01 Mon 02:03:54:064.275 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Adding ReactorMap instance-ip : instance-ip-virtual-machine-interface -> nil 2025-09-01 Mon 02:03:54:064.292 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Updating dependency tacker rules for virtual-machine-interface 2025-09-01 Mon 02:03:54:064.307 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Adding ReactorMap virtual-machine-interface : virtual-machine-interface-route-table -> self 2025-09-01 Mon 02:03:54:064.324 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Adding ReactorMap interface-route-table : self -> virtual-machine-interface-route-table 2025-09-01 Mon 02:03:54:064.348 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Adding ReactorMap interface-route-table : virtual-machine-interface-route-table -> nil 2025-09-01 Mon 02:03:54:064.366 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Updating dependency tacker rules for virtual-machine-interface 2025-09-01 Mon 02:03:54:064.381 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Adding ReactorMap virtual-machine-interface : subnet-virtual-machine-interface -> self 2025-09-01 Mon 02:03:54:064.399 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Adding ReactorMap subnet : self -> subnet-virtual-machine-interface 2025-09-01 Mon 02:03:54:064.414 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Adding ReactorMap subnet : subnet-virtual-machine-interface -> nil 2025-09-01 Mon 02:03:54:064.467 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Updating dependency tacker rules for virtual-machine-interface 2025-09-01 Mon 02:03:54:064.482 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Adding ReactorMap virtual-machine-interface : logical-interface-virtual-machine-interface -> self 2025-09-01 Mon 02:03:54:064.500 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Adding ReactorMap logical-interface : physical-interface-logical-interface -> logical-interface-virtual-machine-interface 2025-09-01 Mon 02:03:54:064.518 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Adding ReactorMap physical-interface : physical-router-physical-interface -> physical-interface-logical-interface 2025-09-01 Mon 02:03:54:064.535 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Adding ReactorMap physical-router : self -> physical-router-physical-interface 2025-09-01 Mon 02:03:54:064.551 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Adding ReactorMap physical-router : physical-router-physical-interface -> nil 2025-09-01 Mon 02:03:54:064.569 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Updating dependency tacker rules for virtual-machine-interface 2025-09-01 Mon 02:03:54:064.584 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Adding ReactorMap virtual-machine-interface : virtual-port-group-virtual-machine-interface -> self 2025-09-01 Mon 02:03:54:064.602 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Adding ReactorMap virtual-port-group : virtual-port-group-physical-interface -> virtual-port-group-virtual-machine-interface 2025-09-01 Mon 02:03:54:064.618 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Adding ReactorMap physical-interface : physical-router-physical-interface -> virtual-port-group-physical-interface 2025-09-01 Mon 02:03:54:064.633 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Adding ReactorMap physical-router : self -> physical-router-physical-interface 2025-09-01 Mon 02:03:54:064.648 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Adding ReactorMap physical-router : physical-router-physical-interface -> nil 2025-09-01 Mon 02:03:54:064.667 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Updating dependency tacker rules for virtual-machine-interface 2025-09-01 Mon 02:03:54:064.682 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Adding ReactorMap virtual-machine-interface : bgpaas-virtual-machine-interface -> self 2025-09-01 Mon 02:03:54:064.698 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Adding ReactorMap bgp-as-a-service : self -> bgpaas-virtual-machine-interface 2025-09-01 Mon 02:03:54:064.721 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Adding ReactorMap bgp-as-a-service : bgpaas-bgp-router -> bgpaas-virtual-machine-interface 2025-09-01 Mon 02:03:54:064.740 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Adding ReactorMap bgp-router : self -> bgpaas-bgp-router 2025-09-01 Mon 02:03:54:064.755 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Adding ReactorMap bgp-router : instance-bgp-router -> bgpaas-bgp-router 2025-09-01 Mon 02:03:54:064.771 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Adding ReactorMap routing-instance : self -> instance-bgp-router 2025-09-01 Mon 02:03:54:064.786 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Adding ReactorMap routing-instance : instance-bgp-router -> nil 2025-09-01 Mon 02:03:54:064.803 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Updating dependency tacker rules for virtual-machine-interface 2025-09-01 Mon 02:03:54:064.818 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Adding ReactorMap virtual-machine-interface : bgpaas-virtual-machine-interface -> self 2025-09-01 Mon 02:03:54:064.833 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Adding ReactorMap bgp-as-a-service : self -> bgpaas-virtual-machine-interface 2025-09-01 Mon 02:03:54:064.849 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Adding ReactorMap bgp-as-a-service : bgpaas-health-check -> bgpaas-virtual-machine-interface 2025-09-01 Mon 02:03:54:064.866 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Adding ReactorMap service-health-check : self -> bgpaas-health-check 2025-09-01 Mon 02:03:54:064.882 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Adding ReactorMap service-health-check : bgpaas-health-check -> nil 2025-09-01 Mon 02:03:54:064.899 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Updating dependency tacker rules for virtual-machine-interface 2025-09-01 Mon 02:03:54:064.914 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Adding ReactorMap virtual-machine-interface : virtual-machine-interface-qos-config -> self 2025-09-01 Mon 02:03:54:064.930 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Adding ReactorMap qos-config : self -> virtual-machine-interface-qos-config 2025-09-01 Mon 02:03:54:064.945 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Adding ReactorMap qos-config : virtual-machine-interface-qos-config -> nil 2025-09-01 Mon 02:03:54:064.963 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Updating dependency tacker rules for virtual-machine-interface 2025-09-01 Mon 02:03:54:064.978 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Adding ReactorMap virtual-machine-interface : virtual-machine-interface-security-logging-object -> self 2025-09-01 Mon 02:03:54:064.994 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Adding ReactorMap security-logging-object : self -> virtual-machine-interface-security-logging-object 2025-09-01 Mon 02:03:54:065.010 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Adding ReactorMap security-logging-object : virtual-machine-interface-security-logging-object -> nil 2025-09-01 Mon 02:03:54:065.029 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Updating dependency tacker rules for virtual-machine-interface 2025-09-01 Mon 02:03:54:065.044 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Adding ReactorMap virtual-machine-interface : virtual-machine-interface-bridge-domain -> self 2025-09-01 Mon 02:03:54:065.069 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Adding ReactorMap virtual-machine-interface-bridge-domain : self -> virtual-machine-interface-bridge-domain 2025-09-01 Mon 02:03:54:065.085 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Adding ReactorMap virtual-machine-interface-bridge-domain : virtual-machine-interface-bridge-domain -> virtual-machine-interface-bridge-domain 2025-09-01 Mon 02:03:54:065.103 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Adding ReactorMap bridge-domain : self -> virtual-machine-interface-bridge-domain 2025-09-01 Mon 02:03:54:065.118 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Adding ReactorMap bridge-domain : virtual-network-bridge-domain -> virtual-machine-interface-bridge-domain 2025-09-01 Mon 02:03:54:065.135 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Adding ReactorMap virtual-network : self -> virtual-network-bridge-domain 2025-09-01 Mon 02:03:54:065.151 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Adding ReactorMap virtual-network : virtual-network-bridge-domain -> nil 2025-09-01 Mon 02:03:54:065.168 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Updating dependency tacker rules for virtual-machine-interface 2025-09-01 Mon 02:03:54:065.183 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Adding ReactorMap virtual-machine-interface : virtual-machine-interface-bridge-domain -> self 2025-09-01 Mon 02:03:54:065.216 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Adding ReactorMap virtual-machine-interface-bridge-domain : self -> virtual-machine-interface-bridge-domain 2025-09-01 Mon 02:03:54:065.231 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Adding ReactorMap virtual-machine-interface-bridge-domain : virtual-machine-interface-bridge-domain -> virtual-machine-interface-bridge-domain 2025-09-01 Mon 02:03:54:065.247 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Adding ReactorMap bridge-domain : self -> virtual-machine-interface-bridge-domain 2025-09-01 Mon 02:03:54:065.262 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Adding ReactorMap bridge-domain : virtual-machine-interface-bridge-domain -> nil 2025-09-01 Mon 02:03:54:065.281 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Updating dependency tacker rules for virtual-machine-interface 2025-09-01 Mon 02:03:54:065.297 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Adding ReactorMap virtual-machine-interface : virtual-machine-interface-tag -> self 2025-09-01 Mon 02:03:54:065.312 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Adding ReactorMap tag : self -> virtual-machine-interface-tag 2025-09-01 Mon 02:03:54:065.328 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Adding ReactorMap tag : application-policy-set-tag -> virtual-machine-interface-tag 2025-09-01 Mon 02:03:54:065.344 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Adding ReactorMap application-policy-set : self -> application-policy-set-tag 2025-09-01 Mon 02:03:54:065.359 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Adding ReactorMap application-policy-set : application-policy-set-firewall-policy -> application-policy-set-tag 2025-09-01 Mon 02:03:54:065.377 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Adding ReactorMap application-policy-set-firewall-policy : self -> application-policy-set-firewall-policy 2025-09-01 Mon 02:03:54:065.400 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Adding ReactorMap application-policy-set-firewall-policy : application-policy-set-firewall-policy -> application-policy-set-firewall-policy 2025-09-01 Mon 02:03:54:065.417 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Adding ReactorMap firewall-policy : self -> application-policy-set-firewall-policy 2025-09-01 Mon 02:03:54:065.433 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Adding ReactorMap firewall-policy : application-policy-set-firewall-policy -> nil 2025-09-01 Mon 02:03:54:065.456 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Updating dependency tacker rules for virtual-machine-interface 2025-09-01 Mon 02:03:54:065.472 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Adding ReactorMap virtual-machine-interface : virtual-machine-interface-virtual-network -> self 2025-09-01 Mon 02:03:54:065.487 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Adding ReactorMap virtual-network : self -> virtual-machine-interface-virtual-network 2025-09-01 Mon 02:03:54:065.504 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Adding ReactorMap virtual-network : virtual-network-tag -> virtual-machine-interface-virtual-network 2025-09-01 Mon 02:03:54:065.520 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Adding ReactorMap tag : self -> virtual-network-tag 2025-09-01 Mon 02:03:54:065.535 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Adding ReactorMap tag : application-policy-set-tag -> virtual-network-tag 2025-09-01 Mon 02:03:54:065.550 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Adding ReactorMap application-policy-set : self -> application-policy-set-tag 2025-09-01 Mon 02:03:54:065.565 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Adding ReactorMap application-policy-set : application-policy-set-firewall-policy -> application-policy-set-tag 2025-09-01 Mon 02:03:54:065.580 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Adding ReactorMap application-policy-set-firewall-policy : self -> application-policy-set-firewall-policy 2025-09-01 Mon 02:03:54:065.595 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Adding ReactorMap application-policy-set-firewall-policy : application-policy-set-firewall-policy -> application-policy-set-firewall-policy 2025-09-01 Mon 02:03:54:065.610 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Adding ReactorMap firewall-policy : self -> application-policy-set-firewall-policy 2025-09-01 Mon 02:03:54:065.625 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Adding ReactorMap firewall-policy : application-policy-set-firewall-policy -> nil 2025-09-01 Mon 02:03:54:065.646 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Updating dependency tacker rules for virtual-machine-interface 2025-09-01 Mon 02:03:54:065.661 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Adding ReactorMap virtual-machine-interface : virtual-machine-interface-virtual-machine -> self 2025-09-01 Mon 02:03:54:065.677 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Adding ReactorMap virtual-machine : self -> virtual-machine-interface-virtual-machine 2025-09-01 Mon 02:03:54:065.692 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Adding ReactorMap virtual-machine : virtual-machine-tag -> virtual-machine-interface-virtual-machine 2025-09-01 Mon 02:03:54:065.708 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Adding ReactorMap tag : self -> virtual-machine-tag 2025-09-01 Mon 02:03:54:065.730 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Adding ReactorMap tag : application-policy-set-tag -> virtual-machine-tag 2025-09-01 Mon 02:03:54:065.746 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Adding ReactorMap application-policy-set : self -> application-policy-set-tag 2025-09-01 Mon 02:03:54:065.761 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Adding ReactorMap application-policy-set : application-policy-set-firewall-policy -> application-policy-set-tag 2025-09-01 Mon 02:03:54:065.776 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Adding ReactorMap application-policy-set-firewall-policy : self -> application-policy-set-firewall-policy 2025-09-01 Mon 02:03:54:065.791 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Adding ReactorMap application-policy-set-firewall-policy : application-policy-set-firewall-policy -> application-policy-set-firewall-policy 2025-09-01 Mon 02:03:54:065.805 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Adding ReactorMap firewall-policy : self -> application-policy-set-firewall-policy 2025-09-01 Mon 02:03:54:065.820 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Adding ReactorMap firewall-policy : application-policy-set-firewall-policy -> nil 2025-09-01 Mon 02:03:54:065.841 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Updating dependency tacker rules for virtual-machine-interface 2025-09-01 Mon 02:03:54:065.856 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Adding ReactorMap virtual-machine-interface : virtual-machine-interface-sub-interface -> self 2025-09-01 Mon 02:03:54:065.871 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Adding ReactorMap virtual-machine-interface : self -> virtual-machine-interface-sub-interface 2025-09-01 Mon 02:03:54:065.887 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Adding ReactorMap virtual-machine-interface : virtual-machine-interface-virtual-machine -> virtual-machine-interface-sub-interface 2025-09-01 Mon 02:03:54:065.902 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Adding ReactorMap virtual-machine : self -> virtual-machine-interface-virtual-machine 2025-09-01 Mon 02:03:54:065.917 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Adding ReactorMap virtual-machine : virtual-machine-tag -> virtual-machine-interface-virtual-machine 2025-09-01 Mon 02:03:54:065.932 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Adding ReactorMap tag : self -> virtual-machine-tag 2025-09-01 Mon 02:03:54:065.947 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Adding ReactorMap tag : application-policy-set-tag -> virtual-machine-tag 2025-09-01 Mon 02:03:54:065.962 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Adding ReactorMap application-policy-set : self -> application-policy-set-tag 2025-09-01 Mon 02:03:54:065.977 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Adding ReactorMap application-policy-set : application-policy-set-firewall-policy -> application-policy-set-tag 2025-09-01 Mon 02:03:54:065.992 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Adding ReactorMap application-policy-set-firewall-policy : self -> application-policy-set-firewall-policy 2025-09-01 Mon 02:03:54:066.006 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Adding ReactorMap application-policy-set-firewall-policy : application-policy-set-firewall-policy -> application-policy-set-firewall-policy 2025-09-01 Mon 02:03:54:066.029 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Adding ReactorMap firewall-policy : self -> application-policy-set-firewall-policy 2025-09-01 Mon 02:03:54:066.044 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Adding ReactorMap firewall-policy : application-policy-set-firewall-policy -> nil 2025-09-01 Mon 02:03:54:066.065 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Updating dependency tacker rules for virtual-machine-interface 2025-09-01 Mon 02:03:54:066.080 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Adding ReactorMap virtual-machine-interface : project-virtual-machine-interface -> self 2025-09-01 Mon 02:03:54:066.097 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Adding ReactorMap project : self -> project-virtual-machine-interface 2025-09-01 Mon 02:03:54:066.113 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Adding ReactorMap project : project-tag -> project-virtual-machine-interface 2025-09-01 Mon 02:03:54:066.129 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Adding ReactorMap tag : self -> project-tag 2025-09-01 Mon 02:03:54:066.144 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Adding ReactorMap tag : application-policy-set-tag -> project-tag 2025-09-01 Mon 02:03:54:066.159 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Adding ReactorMap application-policy-set : self -> application-policy-set-tag 2025-09-01 Mon 02:03:54:066.174 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Adding ReactorMap application-policy-set : application-policy-set-firewall-policy -> application-policy-set-tag 2025-09-01 Mon 02:03:54:066.200 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Adding ReactorMap application-policy-set-firewall-policy : self -> application-policy-set-firewall-policy 2025-09-01 Mon 02:03:54:066.216 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Adding ReactorMap application-policy-set-firewall-policy : application-policy-set-firewall-policy -> application-policy-set-firewall-policy 2025-09-01 Mon 02:03:54:066.231 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Adding ReactorMap firewall-policy : self -> application-policy-set-firewall-policy 2025-09-01 Mon 02:03:54:066.246 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Adding ReactorMap firewall-policy : application-policy-set-firewall-policy -> nil 2025-09-01 Mon 02:03:54:066.265 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Updating dependency tacker rules for virtual-machine-interface 2025-09-01 Mon 02:03:54:066.280 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Adding ReactorMap virtual-machine-interface : port-tuple-interface -> self 2025-09-01 Mon 02:03:54:066.297 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Adding ReactorMap port-tuple : self -> port-tuple-interface 2025-09-01 Mon 02:03:54:066.312 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Adding ReactorMap port-tuple : port-tuple-interface -> port-tuple-interface 2025-09-01 Mon 02:03:54:066.328 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Adding ReactorMap virtual-machine-interface : self -> port-tuple-interface 2025-09-01 Mon 02:03:54:066.344 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Adding ReactorMap virtual-machine-interface : port-tuple-interface -> nil 2025-09-01 Mon 02:03:54:066.361 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Updating dependency tacker rules for virtual-machine-interface 2025-09-01 Mon 02:03:54:066.383 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Adding ReactorMap virtual-machine-interface : logical-router-interface -> self 2025-09-01 Mon 02:03:54:066.400 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Adding ReactorMap logical-router : self -> logical-router-interface 2025-09-01 Mon 02:03:54:066.415 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Adding ReactorMap logical-router : logical-router-interface -> nil 2025-09-01 Mon 02:03:54:066.433 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Updating dependency tacker rules for physical-interface 2025-09-01 Mon 02:03:54:066.449 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Adding ReactorMap physical-interface : physical-router-physical-interface -> self 2025-09-01 Mon 02:03:54:066.464 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Adding ReactorMap physical-router : self -> physical-router-physical-interface 2025-09-01 Mon 02:03:54:066.479 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Adding ReactorMap physical-router : physical-router-physical-interface -> nil 2025-09-01 Mon 02:03:54:066.496 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Updating dependency tacker rules for physical-interface 2025-09-01 Mon 02:03:54:066.511 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Adding ReactorMap physical-interface : virtual-port-group-physical-interface -> self 2025-09-01 Mon 02:03:54:066.529 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Adding ReactorMap virtual-port-group-physical-interface : self -> virtual-port-group-physical-interface 2025-09-01 Mon 02:03:54:066.545 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Adding ReactorMap virtual-port-group-physical-interface : virtual-port-group-physical-interface -> virtual-port-group-physical-interface 2025-09-01 Mon 02:03:54:066.560 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Adding ReactorMap virtual-port-group : self -> virtual-port-group-physical-interface 2025-09-01 Mon 02:03:54:066.576 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Adding ReactorMap virtual-port-group : virtual-port-group-physical-interface -> nil 2025-09-01 Mon 02:03:54:066.593 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Updating dependency tacker rules for logical-interface 2025-09-01 Mon 02:03:54:066.609 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Adding ReactorMap logical-interface : physical-interface-logical-interface -> self 2025-09-01 Mon 02:03:54:066.624 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Adding ReactorMap physical-interface : self -> physical-interface-logical-interface 2025-09-01 Mon 02:03:54:066.639 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Adding ReactorMap physical-interface : physical-router-physical-interface -> physical-interface-logical-interface 2025-09-01 Mon 02:03:54:066.655 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Adding ReactorMap physical-router : self -> physical-router-physical-interface 2025-09-01 Mon 02:03:54:066.670 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Adding ReactorMap physical-router : physical-router-physical-interface -> nil 2025-09-01 Mon 02:03:54:066.687 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Updating dependency tacker rules for logical-interface 2025-09-01 Mon 02:03:54:066.709 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Adding ReactorMap logical-interface : logical-interface-virtual-machine-interface -> self 2025-09-01 Mon 02:03:54:066.725 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Adding ReactorMap virtual-machine-interface : self -> logical-interface-virtual-machine-interface 2025-09-01 Mon 02:03:54:066.740 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Adding ReactorMap virtual-machine-interface : logical-interface-virtual-machine-interface -> nil 2025-09-01 Mon 02:03:54:066.758 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Updating dependency tacker rules for logical-interface 2025-09-01 Mon 02:03:54:066.773 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Adding ReactorMap logical-interface : physical-router-logical-interface -> self 2025-09-01 Mon 02:03:54:066.788 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Adding ReactorMap physical-router : self -> physical-router-logical-interface 2025-09-01 Mon 02:03:54:066.803 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Adding ReactorMap physical-router : physical-router-logical-interface -> nil 2025-09-01 Mon 02:03:54:066.821 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Updating dependency tacker rules for physical-router 2025-09-01 Mon 02:03:54:066.836 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Adding ReactorMap physical-router : physical-router-physical-interface -> self 2025-09-01 Mon 02:03:54:066.851 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Adding ReactorMap physical-interface : self -> physical-router-physical-interface 2025-09-01 Mon 02:03:54:066.867 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Adding ReactorMap physical-interface : physical-router-physical-interface -> nil 2025-09-01 Mon 02:03:54:066.886 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Updating dependency tacker rules for service-health-check 2025-09-01 Mon 02:03:54:066.901 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Adding ReactorMap service-health-check : service-port-health-check -> self 2025-09-01 Mon 02:03:54:066.916 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Adding ReactorMap virtual-machine-interface : self -> service-port-health-check 2025-09-01 Mon 02:03:54:066.932 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Adding ReactorMap virtual-machine-interface : virtual-machine-interface-virtual-network -> service-port-health-check 2025-09-01 Mon 02:03:54:066.948 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Adding ReactorMap virtual-network : virtual-network-network-ipam -> virtual-machine-interface-virtual-network 2025-09-01 Mon 02:03:54:066.963 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Adding ReactorMap virtual-network-network-ipam : self -> virtual-network-network-ipam 2025-09-01 Mon 02:03:54:066.978 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Adding ReactorMap virtual-network-network-ipam : virtual-network-network-ipam -> nil 2025-09-01 Mon 02:03:54:066.997 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Updating dependency tacker rules for service-health-check 2025-09-01 Mon 02:03:54:067.012 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Adding ReactorMap service-health-check : service-port-health-check -> self 2025-09-01 Mon 02:03:54:067.035 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Adding ReactorMap virtual-machine-interface : self -> service-port-health-check 2025-09-01 Mon 02:03:54:067.051 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Adding ReactorMap virtual-machine-interface : port-tuple-interface -> service-port-health-check 2025-09-01 Mon 02:03:54:067.066 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Adding ReactorMap port-tuple : self -> port-tuple-interface 2025-09-01 Mon 02:03:54:067.081 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Adding ReactorMap port-tuple : port-tuple-interface -> port-tuple-interface 2025-09-01 Mon 02:03:54:067.096 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Adding ReactorMap virtual-machine-interface : self -> port-tuple-interface 2025-09-01 Mon 02:03:54:067.112 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Adding ReactorMap virtual-machine-interface : virtual-machine-interface-virtual-network -> port-tuple-interface 2025-09-01 Mon 02:03:54:067.127 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Adding ReactorMap virtual-network : virtual-network-network-ipam -> virtual-machine-interface-virtual-network 2025-09-01 Mon 02:03:54:067.142 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Adding ReactorMap virtual-network-network-ipam : self -> virtual-network-network-ipam 2025-09-01 Mon 02:03:54:067.157 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Adding ReactorMap virtual-network-network-ipam : virtual-network-network-ipam -> nil 2025-09-01 Mon 02:03:54:067.175 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Updating dependency tacker rules for qos-config 2025-09-01 Mon 02:03:54:067.204 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Adding ReactorMap qos-config : global-qos-config-qos-config -> self 2025-09-01 Mon 02:03:54:067.223 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Adding ReactorMap global-qos-config : global-qos-config-qos-config -> nil 2025-09-01 Mon 02:03:54:067.243 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Updating dependency tacker rules for forwarding-class 2025-09-01 Mon 02:03:54:067.258 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Adding ReactorMap forwarding-class : forwarding-class-qos-queue -> self 2025-09-01 Mon 02:03:54:067.275 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Adding ReactorMap qos-queue : self -> forwarding-class-qos-queue 2025-09-01 Mon 02:03:54:067.291 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Adding ReactorMap qos-queue : forwarding-class-qos-queue -> nil 2025-09-01 Mon 02:03:54:067.309 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Updating dependency tacker rules for security-logging-object 2025-09-01 Mon 02:03:54:067.324 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Adding ReactorMap security-logging-object : security-logging-object-network-policy -> self 2025-09-01 Mon 02:03:54:067.341 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Adding ReactorMap network-policy : self -> security-logging-object-network-policy 2025-09-01 Mon 02:03:54:067.356 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Adding ReactorMap network-policy : security-logging-object-network-policy -> nil 2025-09-01 Mon 02:03:54:067.374 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Updating dependency tacker rules for security-logging-object 2025-09-01 Mon 02:03:54:067.396 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Adding ReactorMap security-logging-object : security-logging-object-security-group -> self 2025-09-01 Mon 02:03:54:067.412 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Adding ReactorMap security-group : self -> security-logging-object-security-group 2025-09-01 Mon 02:03:54:067.428 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Adding ReactorMap security-group : security-logging-object-security-group -> nil 2025-09-01 Mon 02:03:54:067.445 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Updating dependency tacker rules for security-logging-object 2025-09-01 Mon 02:03:54:067.460 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Adding ReactorMap security-logging-object : firewall-policy-security-logging-object -> self 2025-09-01 Mon 02:03:54:067.477 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Adding ReactorMap firewall-policy-security-logging-object : self -> firewall-policy-security-logging-object 2025-09-01 Mon 02:03:54:067.493 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Adding ReactorMap firewall-policy-security-logging-object : firewall-policy-security-logging-object -> firewall-policy-security-logging-object 2025-09-01 Mon 02:03:54:067.509 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Adding ReactorMap firewall-policy : firewall-policy-security-logging-object -> nil 2025-09-01 Mon 02:03:54:067.527 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Updating dependency tacker rules for security-logging-object 2025-09-01 Mon 02:03:54:067.542 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Adding ReactorMap security-logging-object : firewall-rule-security-logging-object -> self 2025-09-01 Mon 02:03:54:067.559 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Adding ReactorMap firewall-rule-security-logging-object : self -> firewall-rule-security-logging-object 2025-09-01 Mon 02:03:54:067.574 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Adding ReactorMap firewall-rule-security-logging-object : firewall-rule-security-logging-object -> firewall-rule-security-logging-object 2025-09-01 Mon 02:03:54:067.592 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Adding ReactorMap firewall-rule : firewall-rule-security-logging-object -> nil 2025-09-01 Mon 02:03:54:067.612 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Updating dependency tacker rules for virtual-router 2025-09-01 Mon 02:03:54:067.627 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Adding ReactorMap virtual-router : virtual-router-sub-cluster -> self 2025-09-01 Mon 02:03:54:067.645 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Adding ReactorMap sub-cluster : self -> virtual-router-sub-cluster 2025-09-01 Mon 02:03:54:067.660 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Adding ReactorMap sub-cluster : bgp-router-sub-cluster -> virtual-router-sub-cluster 2025-09-01 Mon 02:03:54:067.676 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Adding ReactorMap bgp-router : self -> bgp-router-sub-cluster 2025-09-01 Mon 02:03:54:067.692 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Adding ReactorMap bgp-router : bgp-router-sub-cluster -> nil 2025-09-01 Mon 02:03:54:067.715 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Updating dependency tacker rules for bridge-domain 2025-09-01 Mon 02:03:54:067.730 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Adding ReactorMap bridge-domain : virtual-network-bridge-domain -> self 2025-09-01 Mon 02:03:54:067.753 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Adding ReactorMap virtual-network : self -> virtual-network-bridge-domain 2025-09-01 Mon 02:03:54:067.769 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Adding ReactorMap virtual-network : virtual-network-routing-instance -> virtual-network-bridge-domain 2025-09-01 Mon 02:03:54:067.785 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Adding ReactorMap routing-instance : self -> virtual-network-routing-instance 2025-09-01 Mon 02:03:54:067.800 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Adding ReactorMap routing-instance : virtual-network-routing-instance -> nil 2025-09-01 Mon 02:03:54:067.816 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Updating dependency tacker rules for bridge-domain 2025-09-01 Mon 02:03:54:067.831 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Adding ReactorMap bridge-domain : virtual-network-bridge-domain -> self 2025-09-01 Mon 02:03:54:067.846 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Adding ReactorMap virtual-network : self -> virtual-network-bridge-domain 2025-09-01 Mon 02:03:54:067.861 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Adding ReactorMap virtual-network : virtual-network-bridge-domain -> nil 2025-09-01 Mon 02:03:54:067.879 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Updating dependency tacker rules for application-policy-set 2025-09-01 Mon 02:03:54:067.894 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Adding ReactorMap application-policy-set : application-policy-set-firewall-policy -> self 2025-09-01 Mon 02:03:54:067.910 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Adding ReactorMap application-policy-set-firewall-policy : self -> application-policy-set-firewall-policy 2025-09-01 Mon 02:03:54:067.925 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Adding ReactorMap application-policy-set-firewall-policy : application-policy-set-firewall-policy -> application-policy-set-firewall-policy 2025-09-01 Mon 02:03:54:067.940 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Adding ReactorMap firewall-policy : self -> application-policy-set-firewall-policy 2025-09-01 Mon 02:03:54:067.955 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Adding ReactorMap firewall-policy : application-policy-set-firewall-policy -> nil 2025-09-01 Mon 02:03:54:067.974 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Updating dependency tacker rules for firewall-policy 2025-09-01 Mon 02:03:54:067.990 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Adding ReactorMap firewall-policy : firewall-policy-firewall-rule -> self 2025-09-01 Mon 02:03:54:068.007 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Adding ReactorMap firewall-policy-firewall-rule : self -> firewall-policy-firewall-rule 2025-09-01 Mon 02:03:54:068.023 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Adding ReactorMap firewall-policy-firewall-rule : firewall-policy-firewall-rule -> firewall-policy-firewall-rule 2025-09-01 Mon 02:03:54:068.038 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Adding ReactorMap firewall-rule : self -> firewall-policy-firewall-rule 2025-09-01 Mon 02:03:54:068.054 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Adding ReactorMap firewall-rule : firewall-rule-service-group -> firewall-policy-firewall-rule 2025-09-01 Mon 02:03:54:068.080 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Adding ReactorMap service-group : self -> firewall-rule-service-group 2025-09-01 Mon 02:03:54:068.096 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Adding ReactorMap service-group : firewall-rule-service-group -> nil 2025-09-01 Mon 02:03:54:068.114 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Updating dependency tacker rules for firewall-policy 2025-09-01 Mon 02:03:54:068.129 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Adding ReactorMap firewall-policy : firewall-policy-firewall-rule -> self 2025-09-01 Mon 02:03:54:068.145 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Adding ReactorMap firewall-policy-firewall-rule : self -> firewall-policy-firewall-rule 2025-09-01 Mon 02:03:54:068.160 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Adding ReactorMap firewall-policy-firewall-rule : firewall-policy-firewall-rule -> firewall-policy-firewall-rule 2025-09-01 Mon 02:03:54:068.175 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Adding ReactorMap firewall-rule : self -> firewall-policy-firewall-rule 2025-09-01 Mon 02:03:54:068.204 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Adding ReactorMap firewall-rule : firewall-rule-address-group -> firewall-policy-firewall-rule 2025-09-01 Mon 02:03:54:068.221 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Adding ReactorMap address-group : self -> firewall-rule-address-group 2025-09-01 Mon 02:03:54:068.237 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Adding ReactorMap address-group : address-group-tag -> firewall-rule-address-group 2025-09-01 Mon 02:03:54:068.253 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Adding ReactorMap tag : self -> address-group-tag 2025-09-01 Mon 02:03:54:068.268 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Adding ReactorMap tag : address-group-tag -> nil 2025-09-01 Mon 02:03:54:068.286 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Updating dependency tacker rules for firewall-policy 2025-09-01 Mon 02:03:54:068.301 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Adding ReactorMap firewall-policy : firewall-policy-firewall-rule -> self 2025-09-01 Mon 02:03:54:068.315 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Adding ReactorMap firewall-policy-firewall-rule : self -> firewall-policy-firewall-rule 2025-09-01 Mon 02:03:54:068.330 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Adding ReactorMap firewall-policy-firewall-rule : firewall-policy-firewall-rule -> firewall-policy-firewall-rule 2025-09-01 Mon 02:03:54:068.345 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Adding ReactorMap firewall-rule : self -> firewall-policy-firewall-rule 2025-09-01 Mon 02:03:54:068.361 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Adding ReactorMap firewall-rule : firewall-rule-tag -> firewall-policy-firewall-rule 2025-09-01 Mon 02:03:54:068.376 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Adding ReactorMap tag : self -> firewall-rule-tag 2025-09-01 Mon 02:03:54:068.392 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Adding ReactorMap tag : firewall-rule-tag -> nil 2025-09-01 Mon 02:03:54:068.408 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Updating dependency tacker rules for firewall-policy 2025-09-01 Mon 02:03:54:068.454 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Adding ReactorMap firewall-policy : firewall-policy-firewall-rule -> self 2025-09-01 Mon 02:03:54:068.472 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Adding ReactorMap firewall-policy-firewall-rule : self -> firewall-policy-firewall-rule 2025-09-01 Mon 02:03:54:068.487 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Adding ReactorMap firewall-policy-firewall-rule : firewall-policy-firewall-rule -> firewall-policy-firewall-rule 2025-09-01 Mon 02:03:54:068.502 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Adding ReactorMap firewall-rule : self -> firewall-policy-firewall-rule 2025-09-01 Mon 02:03:54:068.518 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Adding ReactorMap firewall-rule : firewall-policy-firewall-rule -> nil 2025-09-01 Mon 02:03:54:236.445 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: TCP [SYS_DEBUG]: TcpServerMessageLog: Server 10.20.0.14:8097 Initialization complete src/contrail-common/io/tcp_server.cc 137 2025-09-01 Mon 02:03:54:240.776 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Current receive sock buffer size is 262144 2025-09-01 Mon 02:03:54:241.147 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011144013312, Pid 113882]: KsyncTxQueue CPU pinning policy <>. KsyncTxQueuen not pinned to CPU 2025-09-01 Mon 02:03:54:243.596 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Vrouter family is 34 2025-09-01 Mon 02:03:54:245.120 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Flow table size : 20185088 2025-09-01 Mon 02:03:54:245.255 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Flow table size : 161218560 2025-09-01 Mon 02:03:54:245.275 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: SetHugePages: Bridge table size:20185088 Flow table size:161218560 2025-09-01 Mon 02:03:54:245.311 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Mem mapping hugepage file:/dev/hugepages/bridge size:20185088 2025-09-01 Mon 02:03:54:245.341 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Mem mapped hugepage file:/dev/hugepages/bridge to addr:0x7f56e0800000 2025-09-01 Mon 02:03:54:245.365 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Mem mapping hugepage file:/dev/hugepages/flow size:161218560 2025-09-01 Mon 02:03:54:245.380 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Mem mapped hugepage file:/dev/hugepages/flow to addr:0x7f56d6e00000 2025-09-01 Mon 02:03:54:245.399 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Sending Huge Page configuration to VROUTER 2025-09-01 Mon 02:03:54:307.527 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Huge pages set successfully 2025-09-01 Mon 02:03:54:324.578 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Flow table size : 161218560 2025-09-01 Mon 02:03:54:324.720 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Flow table size : 20185088 2025-09-01 Mon 02:03:54:400.944 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: SANDESH: ROLE : Generator 2025-09-01 Mon 02:03:54:400.977 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: SANDESH: MODULE : contrail-vrouter-agent 2025-09-01 Mon 02:03:54:400.984 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: SANDESH: SOURCE : cn-jenkins-deploy-platform-ansible-os-3993-1 2025-09-01 Mon 02:03:54:400.997 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: SANDESH: NODE TYPE : Compute 2025-09-01 Mon 02:03:54:401.002 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: SANDESH: INSTANCE ID : 0 2025-09-01 Mon 02:03:54:401.008 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: SANDESH: HTTP SERVER PORT : 8085 2025-09-01 Mon 02:03:54:401.015 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: SANDESH: System Log Send Rate Limit: 0 -> 0 2025-09-01 Mon 02:03:54:401.712 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: HTTP Introspect Init 2025-09-01 Mon 02:03:54:401.902 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: TCP [SYS_DEBUG]: TcpServerMessageLog: Server 0.0.0.0:8085 Initialization complete src/contrail-common/io/tcp_server.cc 137 2025-09-01 Mon 02:03:54:401.988 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Sandesh Http Server Port: 8085 2025-09-01 Mon 02:03:54:402.049 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: SANDESH: NOT Writing http_port 8085TO : /tmp/contrail-vrouter-agent.113006.http_port 2025-09-01 Mon 02:03:54:402.057 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: SANDESH: CONNECT TO COLLECTOR: 1 2025-09-01 Mon 02:03:54:402.646 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Idle 2025-09-01 Mon 02:03:54:405.474 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: SANDESH: No Client: 1756692234404391 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3993-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Idle successful_connections = 0 pid = 113882 http_port = 8085 start_time = 1756692234402667 collector_name = collector_ip = 0.0.0.0:0 collector_list= [ [ ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 0, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 1, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 0, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:03:54:405.641 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011152410112, Pid 113882]: Processing scm::EvStart in state Idle 2025-09-01 Mon 02:03:54:405.798 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011152410112, Pid 113882]: Disconnect 2025-09-01 Mon 02:03:54:405.896 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011152410112, Pid 113882]: Processing scm::EvCollectorUpdate in state Disconnect 2025-09-01 Mon 02:03:54:406.273 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011152410112, Pid 113882]: Connect : Start Connect timer 10.0.0.56:8086 2025-09-01 Mon 02:03:54:406.311 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011171994240, Pid 113882]: -(-1) OnSessionEvent TCP Connect Failed 2025-09-01 Mon 02:03:54:406.342 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011171994240, Pid 113882]: -(-1):EvTcpConnectFail 2025-09-01 Mon 02:03:54:406.350 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011152410112, Pid 113882]: Processing scm::EvSandeshSend in state Connect 2025-09-01 Mon 02:03:54:406.380 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011152410112, Pid 113882]: SANDESH: Send FAILED: 1756692234405785 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-3993-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Idle : EvStart -> Disconnect ], ] ] flag_infos= [ [ ] ] description = Collector connection down, No Configuration for self Number of connections:1, Expected: 3 ], ] ] ] 2025-09-01 Mon 02:03:54:406.391 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011152410112, Pid 113882]: Wrong state: Connect for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:03:54:406.401 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011152410112, Pid 113882]: Processing scm::EvSandeshSend in state Connect 2025-09-01 Mon 02:03:54:406.497 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011152410112, Pid 113882]: SANDESH: Send FAILED: 1756692234405881 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3993-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Disconnect successful_connections = 0 pid = 113882 http_port = 8085 start_time = 1756692234402667 collector_name = collector_ip = 0.0.0.0:0 collector_list= [ [ ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 1, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 0, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:03:54:406.519 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011152410112, Pid 113882]: Wrong state: Connect for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:03:54:406.531 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011152410112, Pid 113882]: Processing scm::EvSandeshSend in state Connect 2025-09-01 Mon 02:03:54:406.617 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011152410112, Pid 113882]: SANDESH: Send FAILED: 1756692234405940 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3993-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Disconnect successful_connections = 0 pid = 113882 http_port = 8085 start_time = 1756692234402667 collector_name = collector_ip = 10.0.0.56:8086 collector_list= [ [ (*_iter40) = 10.0.0.56:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 1, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 0, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:03:54:406.633 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011152410112, Pid 113882]: Wrong state: Connect for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:03:54:406.646 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011152410112, Pid 113882]: Processing scm::EvSandeshSend in state Connect 2025-09-01 Mon 02:03:54:406.665 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011152410112, Pid 113882]: SANDESH: Send FAILED: 1756692234405964 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-3993-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.56:8086, ] ] status = Initializing description = Disconnect : EvCollectorUpdate -> Connect ], ] ] flag_infos= [ [ ] ] description = Collector connection down, No Configuration for self Number of connections:1, Expected: 3 ], ] ] ] 2025-09-01 Mon 02:03:54:406.673 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011152410112, Pid 113882]: Wrong state: Connect for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:03:54:406.683 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011152410112, Pid 113882]: Processing scm::EvSandeshSend in state Connect 2025-09-01 Mon 02:03:54:406.756 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011152410112, Pid 113882]: SANDESH: Send FAILED: 1756692234406333 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3993-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Connect successful_connections = 0 pid = 113882 http_port = 8085 start_time = 1756692234402667 collector_name = collector_ip = 10.0.0.56:8086 collector_list= [ [ (*_iter40) = 10.0.0.56:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 1, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 0, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:03:54:406.772 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011152410112, Pid 113882]: Wrong state: Connect for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:03:54:406.784 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011152410112, Pid 113882]: Processing scm::EvTcpConnectFail in state Connect 2025-09-01 Mon 02:03:54:406.793 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011152410112, Pid 113882]: Connect : EvTcpConnectFail 2025-09-01 Mon 02:03:54:406.850 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011152410112, Pid 113882]: TCP [SYS_ERR]: TcpSessionMessageLog: Session -(-1) > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:03:54:406.888 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011152410112, Pid 113882]: Idle 2025-09-01 Mon 02:03:54:406.983 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011152410112, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:54:407.005 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011152410112, Pid 113882]: SANDESH: Send FAILED: 1756692234406818 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-3993-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = Collector connection down, No Configuration for self Number of connections:1, Expected: 3 ], ] ] ] 2025-09-01 Mon 02:03:54:407.014 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011152410112, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:03:54:407.030 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011152410112, Pid 113882]: Processing scm::EvTcpDeleteSession in state Idle 2025-09-01 Mon 02:03:54:407.087 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011152410112, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:54:407.230 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011152410112, Pid 113882]: SANDESH: Send FAILED: 1756692234406970 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3993-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Idle successful_connections = 0 pid = 113882 http_port = 8085 start_time = 1756692234402667 collector_name = collector_ip = 10.0.0.56:8086 collector_list= [ [ (*_iter40) = 10.0.0.56:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 3 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 2, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 5, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:03:54:407.263 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011152410112, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:03:54:407.277 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011152410112, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:54:409.343 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Handle netlink interface message RTM_NEWLINK for interface lo flags 65609 2025-09-01 Mon 02:03:54:409.478 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Handle netlink interface message RTM_NEWLINK for interface ens3 flags 69699 2025-09-01 Mon 02:03:54:409.539 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Handle netlink interface message RTM_NEWLINK for interface ens6 flags 69699 2025-09-01 Mon 02:03:54:409.550 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Handle netlink interface message RTM_NEWLINK for interface docker0 flags 4099 2025-09-01 Mon 02:03:54:409.585 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Handle netlink interface message RTM_NEWLINK for interface pkt1 flags 65601 2025-09-01 Mon 02:03:54:409.594 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Handle netlink interface message RTM_NEWLINK for interface pkt3 flags 65601 2025-09-01 Mon 02:03:54:409.603 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Handle netlink interface message RTM_NEWLINK for interface pkt2 flags 65601 2025-09-01 Mon 02:03:54:409.635 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Handle netlink interface message RTM_NEWLINK for interface vhost0 flags 69699 2025-09-01 Mon 02:03:54:409.644 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Handle netlink interface message RTM_NEWLINK for interface pkt0 flags 69699 2025-09-01 Mon 02:03:54:409.700 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Handle netlink address message RTM_NEWADDR for interface lo 2025-09-01 Mon 02:03:54:409.718 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Handle netlink address message RTM_NEWADDR for interface ens3 2025-09-01 Mon 02:03:54:409.733 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Handle netlink address message RTM_NEWADDR for interface docker0 2025-09-01 Mon 02:03:54:409.749 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Handle netlink address message RTM_NEWADDR for interface vhost0 2025-09-01 Mon 02:03:54:409.776 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Handle netlink address message RTM_NEWADDR for interface lo 2025-09-01 Mon 02:03:54:409.793 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Handle netlink address message RTM_NEWADDR for interface ens3 2025-09-01 Mon 02:03:54:409.808 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Handle netlink address message RTM_NEWADDR for interface vhost0 2025-09-01 Mon 02:03:54:409.823 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Handle netlink address message RTM_NEWADDR for interface pkt0 2025-09-01 Mon 02:03:54:409.889 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Ignoring Netlink route with family 2 table 254 type 1 scope 0 2025-09-01 Mon 02:03:54:409.912 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Handle netlink route message RTM_NEWROUTE : 10.0.0.0/24 Interface ens3 GW 0.0.0.0 2025-09-01 Mon 02:03:54:409.939 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Handle netlink route message RTM_NEWROUTE : 10.0.0.1/32 Interface ens3 GW 0.0.0.0 2025-09-01 Mon 02:03:54:409.966 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Handle netlink route message RTM_NEWROUTE : 10.0.0.2/32 Interface ens3 GW 0.0.0.0 2025-09-01 Mon 02:03:54:409.991 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Handle netlink route message RTM_NEWROUTE : 10.0.0.4/32 Interface ens3 GW 0.0.0.0 2025-09-01 Mon 02:03:54:410.009 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Handle netlink route message RTM_NEWROUTE : 10.20.0.0/24 Interface vhost0 GW 0.0.0.0 2025-09-01 Mon 02:03:54:410.026 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Handle netlink route message RTM_NEWROUTE : 10.20.0.0/24 Interface vhost0 GW 0.0.0.0 2025-09-01 Mon 02:03:54:410.035 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Ignoring Netlink route with family 2 table 254 type 1 scope 0 2025-09-01 Mon 02:03:54:410.051 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Handle netlink route message RTM_NEWROUTE : 172.17.0.0/16 Interface docker0 GW 0.0.0.0 2025-09-01 Mon 02:03:54:410.060 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Ignoring Netlink route with family 2 table 255 type 2 scope 254 2025-09-01 Mon 02:03:54:410.068 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Ignoring Netlink route with family 2 table 255 type 3 scope 253 2025-09-01 Mon 02:03:54:410.093 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Ignoring Netlink route with family 2 table 255 type 2 scope 254 2025-09-01 Mon 02:03:54:410.107 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Ignoring Netlink route with family 2 table 255 type 3 scope 253 2025-09-01 Mon 02:03:54:410.121 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Ignoring Netlink route with family 2 table 255 type 2 scope 254 2025-09-01 Mon 02:03:54:410.135 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Ignoring Netlink route with family 2 table 255 type 2 scope 254 2025-09-01 Mon 02:03:54:410.149 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Ignoring Netlink route with family 2 table 255 type 3 scope 253 2025-09-01 Mon 02:03:54:410.163 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Ignoring Netlink route with family 2 table 255 type 2 scope 254 2025-09-01 Mon 02:03:54:410.177 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Ignoring Netlink route with family 2 table 255 type 3 scope 253 2025-09-01 Mon 02:03:54:410.066 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011152410112, Pid 113882]: SANDESH: Send FAILED: 1756692234407021 AgentVrf [SYS_INFO]: VrfObjectLog: vrf= [ event = Addition name = default-domain:default-project:ip-fabric:__default__ index = 0 ] file = controller/src/vnsw/agent/oper/vrf.cc line = 414 2025-09-01 Mon 02:03:54:410.241 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011152410112, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: VrfObjectLog 2025-09-01 Mon 02:03:54:410.219 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Ignoring Netlink route with family 10 table 254 type 1 scope 0 2025-09-01 Mon 02:03:54:410.264 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011152410112, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:54:410.317 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011152410112, Pid 113882]: SANDESH: Send FAILED: 1756692234407054 AgentVrf [SYS_INFO]: VrfObjectLog: vrf= [ event = Addition name = default-domain:default-project:ip-fabric:ip-fabric index = 1 ] file = controller/src/vnsw/agent/oper/vrf.cc line = 414 2025-09-01 Mon 02:03:54:410.330 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011152410112, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: VrfObjectLog 2025-09-01 Mon 02:03:54:410.285 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Ignoring Netlink route with family 10 table 254 type 1 scope 0 2025-09-01 Mon 02:03:54:410.364 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Ignoring Netlink route with family 10 table 254 type 1 scope 0 2025-09-01 Mon 02:03:54:410.375 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Ignoring Netlink route with family 10 table 254 type 1 scope 0 2025-09-01 Mon 02:03:54:410.385 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Ignoring Netlink route with family 10 table 255 type 2 scope 0 2025-09-01 Mon 02:03:54:410.397 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Ignoring Netlink route with family 10 table 255 type 2 scope 0 2025-09-01 Mon 02:03:54:410.408 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Ignoring Netlink route with family 10 table 255 type 2 scope 0 2025-09-01 Mon 02:03:54:410.419 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Ignoring Netlink route with family 10 table 255 type 5 scope 0 2025-09-01 Mon 02:03:54:410.431 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Ignoring Netlink route with family 10 table 255 type 5 scope 0 2025-09-01 Mon 02:03:54:410.451 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Ignoring Netlink route with family 10 table 255 type 5 scope 0 2025-09-01 Mon 02:03:54:410.463 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Ignoring Netlink route with family 10 table 255 type 5 scope 0 2025-09-01 Mon 02:03:54:438.371 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: TCP [SYS_DEBUG]: TcpServerMessageLog: Server 0.0.0.0:9091 Initialization complete src/contrail-common/io/tcp_server.cc 137 2025-09-01 Mon 02:03:54:438.563 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Router ID configured. Connection to controller initiated 2025-09-01 Mon 02:03:54:445.413 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011135616512, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:54:445.482 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011135616512, Pid 113882]: SANDESH: Send FAILED: 1756692234445305 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.14 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:54:445.504 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011135616512, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:54:445.518 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011135616512, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:54:445.541 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011135616512, Pid 113882]: SANDESH: Send FAILED: 1756692234445399 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-3993-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.14 server_addrs= [ [ (*_iter6) = 10.20.0.14:5269, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.14, Collector connection down, No Configuration for self Number of connections:2, Expected: 3 ], ] ] ] 2025-09-01 Mon 02:03:54:445.547 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011135616512, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:03:54:446.261 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011152410112, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:54:446.303 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011152410112, Pid 113882]: SANDESH: Send FAILED: 1756692234446184 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.14 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:54:446.315 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011152410112, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:54:446.322 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011152410112, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:54:446.347 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011152410112, Pid 113882]: SANDESH: Send FAILED: 1756692234446251 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-3993-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.14 server_addrs= [ [ (*_iter6) = 10.20.0.14:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.14 server_addrs= [ [ (*_iter6) = 10.20.0.14:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.14, XMPP:dns-server:10.20.0.14, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:03:54:446.370 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011152410112, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:03:54:449.547 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011156608512, Pid 113882]: TCP [SYS_DEBUG]: TcpServerMessageLog: Server [fe80::f816:3eff:fe54:bc9f%vhost0]:8097 Initialization complete src/contrail-common/io/tcp_server.cc 137 2025-09-01 Mon 02:03:54:496.411 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011152410112, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:54:496.556 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011152410112, Pid 113882]: SANDESH: Send FAILED: 1756692234496296 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.14 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:54:496.592 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011152410112, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:54:496.624 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011171994240, Pid 113882]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.14 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:03:54:496.628 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011152410112, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:54:496.736 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011152410112, Pid 113882]: SANDESH: Send FAILED: 1756692234496571 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.14 state = Connect event = Connect Event local_port = 37791 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:54:496.757 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011152410112, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:54:497.941 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011135616512, Pid 113882]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:03:54:498.062 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011139814912, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:54:498.133 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011139814912, Pid 113882]: SANDESH: Send FAILED: 1756692234498018 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.14 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:54:498.153 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011139814912, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:54:498.718 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011160806912, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:54:498.800 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011160806912, Pid 113882]: SANDESH: Send FAILED: 1756692234498668 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.14 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:54:498.837 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011160806912, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:54:498.923 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011160806912, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:54:498.936 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011171994240, Pid 113882]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.14 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:03:54:498.974 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011160806912, Pid 113882]: SANDESH: Send FAILED: 1756692234498858 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.14 state = Connect event = Connect Event local_port = 39375 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:54:499.017 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011160806912, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:54:499.042 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011135616512, Pid 113882]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:03:54:499.172 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011156608512, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:54:499.281 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011156608512, Pid 113882]: SANDESH: Send FAILED: 1756692234499086 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.14 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:54:499.307 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011156608512, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:55:407.120 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011148211712, Pid 113882]: Processing scm::EvIdleHoldTimerExpired in state Idle 2025-09-01 Mon 02:03:55:407.728 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011148211712, Pid 113882]: Connect : Start Connect timer 10.0.0.56:8086 2025-09-01 Mon 02:03:55:407.783 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011171994240, Pid 113882]: -(-1) OnSessionEvent TCP Connect Failed 2025-09-01 Mon 02:03:55:407.864 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011171994240, Pid 113882]: -(-1):EvTcpConnectFail 2025-09-01 Mon 02:03:55:407.989 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011148211712, Pid 113882]: Processing scm::EvSandeshSend in state Connect 2025-09-01 Mon 02:03:55:408.106 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011148211712, Pid 113882]: SANDESH: Send FAILED: 1756692235407326 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-3993-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.14 server_addrs= [ [ (*_iter6) = 10.20.0.14:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.14 server_addrs= [ [ (*_iter6) = 10.20.0.14:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.56:8086, ] ] status = Initializing description = Idle : EvIdleHoldTimerExpired -> Connect ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.14, XMPP:dns-server:10.20.0.14, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:03:55:408.131 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011148211712, Pid 113882]: Wrong state: Connect for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:03:55:408.175 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011148211712, Pid 113882]: Processing scm::EvTcpConnectFail in state Connect 2025-09-01 Mon 02:03:55:408.218 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011148211712, Pid 113882]: Connect : EvTcpConnectFail 2025-09-01 Mon 02:03:55:408.334 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011148211712, Pid 113882]: TCP [SYS_ERR]: TcpSessionMessageLog: Session -(-1) > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:03:55:408.409 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011148211712, Pid 113882]: Idle 2025-09-01 Mon 02:03:55:408.658 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011148211712, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:55:408.974 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011148211712, Pid 113882]: SANDESH: Send FAILED: 1756692235407917 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3993-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Connect successful_connections = 0 pid = 113882 http_port = 8085 start_time = 1756692234402667 collector_name = collector_ip = 10.0.0.56:8086 collector_list= [ [ (*_iter40) = 10.0.0.56:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 5 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 4 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 8 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 8, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 19, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:03:55:409.032 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011148211712, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:03:55:409.064 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011148211712, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:55:409.114 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011148211712, Pid 113882]: SANDESH: Send FAILED: 1756692235408277 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-3993-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.14 server_addrs= [ [ (*_iter6) = 10.20.0.14:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.14 server_addrs= [ [ (*_iter6) = 10.20.0.14:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.14, XMPP:dns-server:10.20.0.14, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:03:55:409.142 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011148211712, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:03:55:409.166 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011148211712, Pid 113882]: Processing scm::EvTcpDeleteSession in state Idle 2025-09-01 Mon 02:03:55:409.260 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011148211712, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:55:409.539 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011148211712, Pid 113882]: SANDESH: Send FAILED: 1756692235408628 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3993-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Idle successful_connections = 0 pid = 113882 http_port = 8085 start_time = 1756692234402667 collector_name = collector_ip = 10.0.0.56:8086 collector_list= [ [ (*_iter40) = 10.0.0.56:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 6 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 4 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 4 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 8 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 9, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 20, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:03:55:409.588 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011148211712, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:03:55:416.337 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011135616512, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:55:416.495 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011135616512, Pid 113882]: SANDESH: Send FAILED: 1756692235416278 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.14 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:55:416.526 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011135616512, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:55:416.641 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011156608512, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:55:416.678 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011171994240, Pid 113882]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.14 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:03:55:416.709 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011156608512, Pid 113882]: SANDESH: Send FAILED: 1756692235416600 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.14 state = Connect event = Connect Event local_port = 44059 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:55:416.800 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011156608512, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:55:416.891 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011139814912, Pid 113882]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:03:55:417.142 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:55:417.258 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: SANDESH: Send FAILED: 1756692235417023 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.14 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:55:417.285 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:55:462.435 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:55:462.569 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011171994240, Pid 113882]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.14 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:03:55:462.601 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: SANDESH: Send FAILED: 1756692235462371 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.14 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:55:462.644 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:55:462.666 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011148211712, Pid 113882]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:03:55:462.682 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:55:462.750 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: SANDESH: Send FAILED: 1756692235462525 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.14 state = Connect event = Connect Event local_port = 33617 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:55:462.780 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:55:462.821 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:55:462.872 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: SANDESH: Send FAILED: 1756692235462704 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.14 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:55:462.903 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:57:298.940 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011148211712, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:57:299.070 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011148211712, Pid 113882]: SANDESH: Send FAILED: 1756692237298875 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.14 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:57:299.099 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011148211712, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:57:299.175 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011135616512, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:57:299.176 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011171994240, Pid 113882]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.14 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:03:57:299.273 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011135616512, Pid 113882]: SANDESH: Send FAILED: 1756692237299125 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.14 state = Connect event = Connect Event local_port = 42585 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:57:299.338 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011135616512, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:57:299.350 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011139814912, Pid 113882]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:03:57:299.643 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011148211712, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:57:299.680 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011148211712, Pid 113882]: SANDESH: Send FAILED: 1756692237299600 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.14 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:57:299.691 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011148211712, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:57:408.589 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011160806912, Pid 113882]: Processing scm::EvIdleHoldTimerExpired in state Idle 2025-09-01 Mon 02:03:57:408.991 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011160806912, Pid 113882]: Connect : Start Connect timer 10.0.0.56:8086 2025-09-01 Mon 02:03:57:409.062 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011171994240, Pid 113882]: -(-1) OnSessionEvent TCP Connect Failed 2025-09-01 Mon 02:03:57:409.125 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011160806912, Pid 113882]: Processing scm::EvSandeshSend in state Connect 2025-09-01 Mon 02:03:57:409.130 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011171994240, Pid 113882]: -(-1):EvTcpConnectFail 2025-09-01 Mon 02:03:57:409.169 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011160806912, Pid 113882]: SANDESH: Send FAILED: 1756692237408719 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-3993-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.14 server_addrs= [ [ (*_iter6) = 10.20.0.14:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.14 server_addrs= [ [ (*_iter6) = 10.20.0.14:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.56:8086, ] ] status = Initializing description = Idle : EvIdleHoldTimerExpired -> Connect ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.14, XMPP:dns-server:10.20.0.14, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:03:57:409.208 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011160806912, Pid 113882]: Wrong state: Connect for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:03:57:409.224 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011160806912, Pid 113882]: Processing scm::EvSandeshSend in state Connect 2025-09-01 Mon 02:03:57:409.447 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011160806912, Pid 113882]: SANDESH: Send FAILED: 1756692237409102 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3993-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Connect successful_connections = 0 pid = 113882 http_port = 8085 start_time = 1756692234402667 collector_name = collector_ip = 10.0.0.56:8086 collector_list= [ [ (*_iter40) = 10.0.0.56:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 7 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 6 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 7 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 17 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 5 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 15, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 32, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:03:57:409.472 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011160806912, Pid 113882]: Wrong state: Connect for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:03:57:409.487 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011160806912, Pid 113882]: Processing scm::EvTcpConnectFail in state Connect 2025-09-01 Mon 02:03:57:409.499 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011160806912, Pid 113882]: Connect : EvTcpConnectFail 2025-09-01 Mon 02:03:57:409.559 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011160806912, Pid 113882]: TCP [SYS_ERR]: TcpSessionMessageLog: Session -(-1) > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:03:57:409.599 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011160806912, Pid 113882]: Idle 2025-09-01 Mon 02:03:57:409.674 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011160806912, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:57:409.704 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011160806912, Pid 113882]: SANDESH: Send FAILED: 1756692237409528 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-3993-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.14 server_addrs= [ [ (*_iter6) = 10.20.0.14:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.14 server_addrs= [ [ (*_iter6) = 10.20.0.14:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.14, XMPP:dns-server:10.20.0.14, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:03:57:409.721 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011160806912, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:03:57:409.734 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011160806912, Pid 113882]: Processing scm::EvTcpDeleteSession in state Idle 2025-09-01 Mon 02:03:57:409.778 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011160806912, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:57:409.951 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011160806912, Pid 113882]: SANDESH: Send FAILED: 1756692237409659 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3993-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Idle successful_connections = 0 pid = 113882 http_port = 8085 start_time = 1756692234402667 collector_name = collector_ip = 10.0.0.56:8086 collector_list= [ [ (*_iter40) = 10.0.0.56:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 8 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 7 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 8 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 17 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 5 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 16, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 34, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:03:57:409.975 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011160806912, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:03:57:433.402 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011156608512, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:57:433.477 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011156608512, Pid 113882]: SANDESH: Send FAILED: 1756692237433352 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.14 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:57:433.490 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011156608512, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:57:433.916 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011171994240, Pid 113882]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.14 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:03:57:433.992 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011135616512, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:57:434.027 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:03:57:434.136 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011135616512, Pid 113882]: SANDESH: Send FAILED: 1756692237433910 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.14 state = Connect event = Connect Event local_port = 42719 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:57:434.176 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011135616512, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:57:434.240 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011135616512, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:57:434.287 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011135616512, Pid 113882]: SANDESH: Send FAILED: 1756692237434152 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.14 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:57:434.314 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011135616512, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:01:043.922 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:01:044.102 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: SANDESH: Send FAILED: 1756692241043854 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.14 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:01:044.211 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:01:044.257 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:01:044.266 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011171994240, Pid 113882]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.14 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:04:01:044.318 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: SANDESH: Send FAILED: 1756692241044149 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.14 state = Connect event = Connect Event local_port = 33147 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:01:044.516 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011148211712, Pid 113882]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:04:01:044.525 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:01:044.630 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:01:044.678 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: SANDESH: Send FAILED: 1756692241044579 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.14 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:01:044.772 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:01:044.812 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:01:044.858 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: SANDESH: Send FAILED: 1756692241044586 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.14 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:01:044.878 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:01:044.899 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:01:044.956 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: SANDESH: Send FAILED: 1756692241044688 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-3993-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.14 server_addrs= [ [ (*_iter6) = 10.20.0.14:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.14 server_addrs= [ [ (*_iter6) = 10.20.0.14:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.14, XMPP:dns-server:10.20.0.14, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:04:01:044.976 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:04:01:394.486 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011152410112, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:01:394.599 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011152410112, Pid 113882]: SANDESH: Send FAILED: 1756692241394430 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.14 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:01:394.626 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011152410112, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:01:394.651 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011152410112, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:01:394.686 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011152410112, Pid 113882]: SANDESH: Send FAILED: 1756692241394619 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.14 state = Connect event = Connect Event local_port = 37781 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:01:394.704 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011152410112, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:01:394.714 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011171994240, Pid 113882]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.14 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:04:01:394.851 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:04:01:394.925 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011152410112, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:01:394.961 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011152410112, Pid 113882]: SANDESH: Send FAILED: 1756692241394904 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.14 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:01:394.979 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011152410112, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:01:394.999 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011152410112, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:01:395.025 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011152410112, Pid 113882]: SANDESH: Send FAILED: 1756692241394914 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.14 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:01:395.052 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011152410112, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:01:395.146 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011139814912, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:01:395.204 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011139814912, Pid 113882]: SANDESH: Send FAILED: 1756692241395087 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-3993-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.14 server_addrs= [ [ (*_iter6) = 10.20.0.14:5269, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.14 server_addrs= [ [ (*_iter6) = 10.20.0.14:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.14, XMPP:dns-server:10.20.0.14, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:04:01:395.215 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011139814912, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:04:01:409.877 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Processing scm::EvIdleHoldTimerExpired in state Idle 2025-09-01 Mon 02:04:01:410.162 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Connect : Start Connect timer 10.0.0.56:8086 2025-09-01 Mon 02:04:01:410.262 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011171994240, Pid 113882]: -(-1) OnSessionEvent TCP Connect Failed 2025-09-01 Mon 02:04:01:410.272 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Processing scm::EvSandeshSend in state Connect 2025-09-01 Mon 02:04:01:410.355 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011171994240, Pid 113882]: -(-1):EvTcpConnectFail 2025-09-01 Mon 02:04:01:410.399 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: SANDESH: Send FAILED: 1756692241409973 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-3993-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.14 server_addrs= [ [ (*_iter6) = 10.20.0.14:5269, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.14 server_addrs= [ [ (*_iter6) = 10.20.0.14:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.56:8086, ] ] status = Initializing description = Idle : EvIdleHoldTimerExpired -> Connect ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.14, XMPP:dns-server:10.20.0.14, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:04:01:410.461 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Wrong state: Connect for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:04:01:410.478 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Processing scm::EvSandeshSend in state Connect 2025-09-01 Mon 02:04:01:410.596 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: SANDESH: Send FAILED: 1756692241410253 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3993-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Connect successful_connections = 0 pid = 113882 http_port = 8085 start_time = 1756692234402667 collector_name = collector_ip = 10.0.0.56:8086 collector_list= [ [ (*_iter40) = 10.0.0.56:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 11 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 8 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 11 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 28 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 8 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 22, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 49, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:04:01:410.615 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Wrong state: Connect for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:04:01:410.627 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Processing scm::EvTcpConnectFail in state Connect 2025-09-01 Mon 02:04:01:410.635 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Connect : EvTcpConnectFail 2025-09-01 Mon 02:04:01:410.684 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: TCP [SYS_ERR]: TcpSessionMessageLog: Session -(-1) > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:04:01:410.721 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Idle 2025-09-01 Mon 02:04:01:410.783 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:01:410.806 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: SANDESH: Send FAILED: 1756692241410660 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-3993-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.14 server_addrs= [ [ (*_iter6) = 10.20.0.14:5269, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.14 server_addrs= [ [ (*_iter6) = 10.20.0.14:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.14, XMPP:dns-server:10.20.0.14, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:04:01:410.818 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:04:01:410.827 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Processing scm::EvTcpDeleteSession in state Idle 2025-09-01 Mon 02:04:01:410.857 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:01:411.007 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: SANDESH: Send FAILED: 1756692241410771 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3993-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Idle successful_connections = 0 pid = 113882 http_port = 8085 start_time = 1756692234402667 collector_name = collector_ip = 10.0.0.56:8086 collector_list= [ [ (*_iter40) = 10.0.0.56:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 12 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 9 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 12 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 28 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 8 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 23, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 51, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:04:01:411.025 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:04:07:666.076 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011171994240, Pid 113882]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.0.0.56:8085::10.0.0.56:58218 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 255 2025-09-01 Mon 02:04:08:604.852 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011148211712, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:08:604.978 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011148211712, Pid 113882]: SANDESH: Send FAILED: 1756692248604803 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.14 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:08:605.011 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011148211712, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:08:605.112 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011135616512, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:08:605.170 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011171994240, Pid 113882]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.14 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:04:08:605.182 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011135616512, Pid 113882]: SANDESH: Send FAILED: 1756692248605066 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.14 state = Connect event = Connect Event local_port = 42851 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:08:605.293 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011135616512, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:08:605.333 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011152410112, Pid 113882]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:04:08:605.410 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011148211712, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:08:605.457 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011148211712, Pid 113882]: SANDESH: Send FAILED: 1756692248605386 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.14 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:08:605.477 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011148211712, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:08:605.504 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011148211712, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:08:605.532 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011148211712, Pid 113882]: SANDESH: Send FAILED: 1756692248605398 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.14 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:08:605.551 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011148211712, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:09:171.105 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011160806912, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:09:171.243 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011160806912, Pid 113882]: SANDESH: Send FAILED: 1756692249171041 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.14 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:09:171.271 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011160806912, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:09:171.303 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011160806912, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:09:171.314 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011171994240, Pid 113882]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.14 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:04:09:171.346 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011160806912, Pid 113882]: SANDESH: Send FAILED: 1756692249171246 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.14 state = Connect event = Connect Event local_port = 42905 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:09:171.537 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011160806912, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:09:171.545 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:04:09:171.661 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011152410112, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:09:171.756 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011152410112, Pid 113882]: SANDESH: Send FAILED: 1756692249171617 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.14 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:09:171.783 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011152410112, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:09:171.808 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011152410112, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:09:171.838 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011152410112, Pid 113882]: SANDESH: Send FAILED: 1756692249171623 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.14 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:09:171.856 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011152410112, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:09:410.949 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011135616512, Pid 113882]: Processing scm::EvIdleHoldTimerExpired in state Idle 2025-09-01 Mon 02:04:09:411.595 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011135616512, Pid 113882]: Connect : Start Connect timer 10.0.0.56:8086 2025-09-01 Mon 02:04:09:411.647 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011171994240, Pid 113882]: -(-1) OnSessionEvent TCP Connect Failed 2025-09-01 Mon 02:04:09:411.734 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011171994240, Pid 113882]: -(-1):EvTcpConnectFail 2025-09-01 Mon 02:04:09:411.937 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011135616512, Pid 113882]: Processing scm::EvSandeshSend in state Connect 2025-09-01 Mon 02:04:09:412.047 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011135616512, Pid 113882]: SANDESH: Send FAILED: 1756692249411131 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-3993-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.14 server_addrs= [ [ (*_iter6) = 10.20.0.14:5269, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.14 server_addrs= [ [ (*_iter6) = 10.20.0.14:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.56:8086, ] ] status = Initializing description = Idle : EvIdleHoldTimerExpired -> Connect ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.14, XMPP:dns-server:10.20.0.14, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:04:09:412.077 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011135616512, Pid 113882]: Wrong state: Connect for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:04:09:412.136 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011135616512, Pid 113882]: Processing scm::EvTcpConnectFail in state Connect 2025-09-01 Mon 02:04:09:412.167 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011135616512, Pid 113882]: Connect : EvTcpConnectFail 2025-09-01 Mon 02:04:09:412.335 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011135616512, Pid 113882]: TCP [SYS_ERR]: TcpSessionMessageLog: Session -(-1) > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:04:09:412.409 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011135616512, Pid 113882]: Idle 2025-09-01 Mon 02:04:09:412.724 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011135616512, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:09:413.061 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011135616512, Pid 113882]: SANDESH: Send FAILED: 1756692249411832 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3993-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Connect successful_connections = 0 pid = 113882 http_port = 8085 start_time = 1756692234402667 collector_name = collector_ip = 10.0.0.56:8086 collector_list= [ [ (*_iter40) = 10.0.0.56:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 13 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 10 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 15 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 36 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 10 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 28, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 61, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:04:09:413.113 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011135616512, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:04:09:413.144 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011135616512, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:09:413.221 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011135616512, Pid 113882]: SANDESH: Send FAILED: 1756692249412267 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-3993-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.14 server_addrs= [ [ (*_iter6) = 10.20.0.14:5269, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.14 server_addrs= [ [ (*_iter6) = 10.20.0.14:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.14, XMPP:dns-server:10.20.0.14, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:04:09:413.244 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011135616512, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:04:09:413.290 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011135616512, Pid 113882]: Processing scm::EvTcpDeleteSession in state Idle 2025-09-01 Mon 02:04:09:413.395 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011135616512, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:09:413.704 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011135616512, Pid 113882]: SANDESH: Send FAILED: 1756692249412690 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3993-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Idle successful_connections = 0 pid = 113882 http_port = 8085 start_time = 1756692234402667 collector_name = collector_ip = 10.0.0.56:8086 collector_list= [ [ (*_iter40) = 10.0.0.56:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 14 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 10 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 16 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 36 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 10 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 29, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 62, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:04:09:413.761 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011135616512, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:04:20:594.429 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011171994240, Pid 113882]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.0.0.56:8085::10.0.0.56:43128 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 255 2025-09-01 Mon 02:04:24:046.985 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011139814912, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:24:048.143 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011139814912, Pid 113882]: SANDESH: Send FAILED: 1756692264046895 [SYS_NOTICE]: UveVrouterAgent: data= [ name = cn-jenkins-deploy-platform-ansible-os-3993-1 interface_list= [ [ ] ] error_intf_list= [ [ ] ] no_config_intf_list= [ [ (*_iter151) = 00000000-0000-0000-0000-000000000000, ] ] vmi_count= [ active = 1 ] down_interface_count = 1 ] 2025-09-01 Mon 02:04:24:048.224 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011139814912, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: UveVrouterAgent 2025-09-01 Mon 02:04:24:048.259 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011139814912, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:24:049.057 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011139814912, Pid 113882]: SANDESH: Send FAILED: 1756692264047223 [SYS_NOTICE]: UveVirtualNetworkAgentTrace: data= [ name = __UNKNOWN__ vrf_stats_list= [ [ [ name = __untitled__ diag_packet_count = 0 unknown_unicast_floods = 0 arp_packet_counts= [ from_vm_interface= [ stats= [ proxies = 0 stitches = 0 floods = 0 ] ] from_physical_interface= [ stats= [ proxies = 0 stitches = 0 floods = 0 ] ] ] offload_packet_counts= [ gro = 0 ] nh_packet_counts= [ discards = 0 resolves = 0 l3_receives = 0 l2_receives = 0 local_vm_l3_forwards = 0 local_vm_l2_forwards = 0 vrf_translates = 0 ecmp_forwards = 0 tunnel_nh_stats= [ udp_encaps = 0 mpls_over_udp_encaps = 0 mpls_over_gre_encaps = 0 vxlan_encaps = 0 ] comp_nh_stats= [ source_replication_forwards = 0 edge_replication_forwards = 0 local_vm_l3_forwards = 0 total_multicast_forwards = 0 ] ] ], ] ] ] 2025-09-01 Mon 02:04:24:049.106 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011139814912, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: UveVirtualNetworkAgentTrace 2025-09-01 Mon 02:04:24:406.249 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:24:406.459 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: SANDESH: Send FAILED: 1756692264405842 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3993-1:Compute:contrail-vrouter-agent:0 sm_queue_count = 0 sm_stats= [ ev_stats= [ [ [ event = scm::EvCollectorUpdate enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvIdleHoldTimerExpired enqueues = 4 dequeues = 4 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvSandeshSend enqueues = 67 dequeues = 67 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvStart enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpConnectFail enqueues = 5 dequeues = 5 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpDeleteSession enqueues = 5 dequeues = 5 enqueue_fails = 0 dequeue_fails = 0 ], [ event = enqueues = 83 dequeues = 83 enqueue_fails = 0 dequeue_fails = 0 ], ] ] state = Idle state_since = 1756692249412406 last_state = Idle last_event = scm::EvSandeshSend last_event_at = 1756692264048254 ] max_sm_queue_count = 5 ] 2025-09-01 Mon 02:04:24:406.539 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:04:24:406.575 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:24:407.106 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: SANDESH: Send FAILED: 1756692264406077 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3993-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Idle successful_connections = 0 pid = 113882 http_port = 8085 start_time = 1756692234402667 collector_name = collector_ip = 10.0.0.56:8086 collector_list= [ [ (*_iter40) = 10.0.0.56:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 15 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 12 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 17 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 36 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 10 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 30, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 67, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:04:24:407.173 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:04:24:411.315 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011135616512, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:24:411.642 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011135616512, Pid 113882]: SANDESH: Send FAILED: 1756692264411143 [SYS_NOTICE]: UveVrouterAgent: data= [ name = cn-jenkins-deploy-platform-ansible-os-3993-1 xmpp_peer_list= [ [ [ ip = 10.20.0.14 status = 0 setup_time = 0 primary = 1 ], ] ] self_ip_list= [ [ (*_iter146) = 10.20.0.14, ] ] build_info = {"build-info":[{"build-version":"v0+nightly","build-time":"2025-09-01 00:13:09.128962","build-user":"root","build-hostname":"jenkins-build-prepare-sandbox-10717-1.novalocal","build-id":"unknown","build-number":"unknown"}]} phy_if= [ [ [ name = ens6 mac_address = fa:16:3e:54:bc:9f ], ] ] vhost_if= [ name = vhost0 mac_address = fa:16:3e:54:bc:9f ] log_file = /var/log/contrail/vrouter-agent/contrail-vrouter-agent.log config_file = /etc/contrail/contrail-vrouter-agent.conf log_local = 1 log_level = SYS_DEBUG log_category = sandesh_http_port = 8085 dns_servers= [ [ (*_iter149) = 10.20.0.14, ] ] vhost_cfg= [ name = vhost0 ip = 10.20.0.14 ip_prefix_len = 24 gateway = 0.0.0.0 ] eth_name= [ [ (*_iter150) = ens6, ] ] tunnel_type = MPLSoGRE hypervisor = kvm control_ip = 10.20.0.14 dns_server_list_cfg= [ [ (*_iter152) = 10.20.0.14:53, ] ] control_node_list_cfg= [ [ (*_iter153) = 10.20.0.14:5269, ] ] hostname_cfg = cn-jenkins-deploy-platform-ansible-os-3993-1 flow_cache_timeout_cfg = 0 ll_max_system_flows_cfg = 16384 ll_max_vm_flows_cfg = 16384 gateway_cfg_list= [ [ ] ] headless_mode_cfg = 1 collector_server_list_cfg= [ [ (*_iter155) = 10.0.0.56:8086, ] ] log_flow = 0 mode = VROUTER platform = HOST vr_limits= [ max_labels = 5120 max_nexthops = 524288 max_interfaces = 4355 max_vrfs = 4096 max_mirror_entries = 255 vrouter_max_bridge_entries = 262144 vrouter_max_oflow_bridge_entries = 53248 vrouter_build_info = {"build-info": [{"build-version": "v0+nightly", "build-time": "2025-09-01 00:11:15.602301", "build-user": "root", "build-hostname": "jenkins-build-prepare-sandbox-10717-1.novalocal"}]} vrouter_max_flow_entries = 524288 vrouter_max_oflow_entries = 105472 vrouter_priority_tagging = 1 ] bgpaas_enabled = 0 port_mirror_enabled = 0 subcluster_name = vr_high_watermark = 80 vr_low_watermark = 75 vr_limit_exceeded_list= [ [ ] ] res_limit = 0 res_table_limit = 0 loopback_ip = 0.0.0.0 gateway_list= [ [ (*_iter161) = 0.0.0.0, ] ] ] 2025-09-01 Mon 02:04:24:411.717 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011135616512, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: UveVrouterAgent 2025-09-01 Mon 02:04:24:411.754 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011135616512, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:24:412.099 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011135616512, Pid 113882]: SANDESH: Send FAILED: 1756692264411329 [SYS_NOTICE]: VrouterStats: data= [ name = cn-jenkins-deploy-platform-ansible-os-3993-1 in_tpkts = 20 in_bytes = 924 out_tpkts = 19 out_bytes = 854 exception_packets = 8 exception_packets_dropped = 8 total_flows = 0 aged_flows = 0 uptime = 1756692234043667 phy_if_info= [ [ _iter254->first = ens6 [ speed = -1 duplexity = 255 ], ] ] raw_vhost_stats= [ name = vhost0 in_pkts = 20 in_bytes = 924 out_pkts = 19 out_bytes = 854 speed = -1 duplexity = -1 ] raw_vhost_drop_stats= [ ds_discard = 0 ds_pull = 0 ds_invalid_if = 0 ds_invalid_arp = 0 ds_trap_no_if = 0 ds_nowhere_to_go = 0 ds_flow_queue_limit_exceeded = 0 ds_flow_no_memory = 0 ds_flow_invalid_protocol = 0 ds_flow_nat_no_rflow = 0 ds_flow_action_drop = 0 ds_flow_action_invalid = 0 ds_flow_unusable = 0 ds_flow_table_full = 0 ds_interface_tx_discard = 0 ds_interface_drop = 0 ds_duplicated = 0 ds_push = 0 ds_ttl_exceeded = 0 ds_invalid_nh = 0 ds_invalid_label = 0 ds_invalid_protocol = 0 ds_interface_rx_discard = 0 ds_invalid_mcast_source = 0 ds_head_alloc_fail = 0 ds_pcow_fail = 0 ds_mcast_clone_fail = 0 ds_rewrite_fail = 0 ds_misc = 0 ds_invalid_packet = 0 ds_cksum_err = 0 ds_no_fmd = 0 ds_invalid_vnid = 0 ds_frag_err = 0 ds_invalid_source = 0 ds_mcast_df_bit = 0 ds_l2_no_route = 0 ds_vlan_fwd_tx = 0 ds_vlan_fwd_enq = 0 ds_drop_new_flow = 0 ds_flow_evict = 0 ds_trap_original = 0 ds_no_memory = 0 ds_fragment_queue_fail = 0 ds_no_frag_entry = 0 ds_icmp_error = 0 ds_clone_fail = 0 ds_drop_pkts = 0 ] raw_drop_stats= [ ds_discard = 0 ds_pull = 0 ds_invalid_if = 0 ds_invalid_arp = 0 ds_trap_no_if = 0 ds_nowhere_to_go = 0 ds_flow_queue_limit_exceeded = 0 ds_flow_no_memory = 0 ds_flow_invalid_protocol = 0 ds_flow_nat_no_rflow = 0 ds_flow_action_drop = 0 ds_flow_action_invalid = 0 ds_flow_unusable = 0 ds_flow_table_full = 0 ds_interface_tx_discard = 0 ds_interface_drop = 0 ds_duplicated = 0 ds_push = 0 ds_ttl_exceeded = 0 ds_invalid_nh = 0 ds_invalid_label = 0 ds_invalid_protocol = 0 ds_interface_rx_discard = 0 ds_invalid_mcast_source = 0 ds_head_alloc_fail = 0 ds_pcow_fail = 0 ds_mcast_clone_fail = 0 ds_rewrite_fail = 0 ds_misc = 0 ds_invalid_packet = 0 ds_cksum_err = 0 ds_no_fmd = 0 ds_invalid_vnid = 0 ds_frag_err = 0 ds_invalid_source = 0 ds_mcast_df_bit = 0 ds_l2_no_route = 0 ds_vlan_fwd_tx = 0 ds_vlan_fwd_enq = 0 ds_drop_new_flow = 0 ds_flow_evict = 0 ds_trap_original = 0 ds_no_memory = 0 ds_fragment_queue_fail = 0 ds_no_frag_entry = 0 ds_icmp_error = 0 ds_clone_fail = 0 ds_drop_pkts = 0 ] flow_rate= [ added_flows = 0 max_flow_adds_per_second = 0 min_flow_adds_per_second = 0 deleted_flows = 0 max_flow_deletes_per_second = 0 min_flow_deletes_per_second = 0 active_flows = 0 hold_flows = 0 ] raw_phy_if_stats= [ [ _iter262->first = ens6 [ in_pkts = 2 in_bytes = 140 out_pkts = 20 out_bytes = 924 ], ] ] raw_phy_if_drop_stats= [ [ _iter263->first = ens6 [ ds_discard = 0 ds_pull = 0 ds_invalid_if = 0 ds_invalid_arp = 0 ds_trap_no_if = 0 ds_nowhere_to_go = 0 ds_flow_queue_limit_exceeded = 0 ds_flow_no_memory = 0 ds_flow_invalid_protocol = 0 ds_flow_nat_no_rflow = 0 ds_flow_action_drop = 0 ds_flow_action_invalid = 0 ds_flow_unusable = 0 ds_flow_table_full = 0 ds_interface_tx_discard = 0 ds_interface_drop = 0 ds_duplicated = 0 ds_push = 0 ds_ttl_exceeded = 0 ds_invalid_nh = 0 ds_invalid_label = 0 ds_invalid_protocol = 0 ds_interface_rx_discard = 0 ds_invalid_mcast_source = 0 ds_head_alloc_fail = 0 ds_pcow_fail = 0 ds_mcast_clone_fail = 0 ds_rewrite_fail = 0 ds_misc = 0 ds_invalid_packet = 0 ds_cksum_err = 0 ds_no_fmd = 0 ds_invalid_vnid = 0 ds_frag_err = 0 ds_invalid_source = 0 ds_mcast_df_bit = 0 ds_l2_no_route = 0 ds_vlan_fwd_tx = 0 ds_vlan_fwd_enq = 0 ds_drop_new_flow = 0 ds_flow_evict = 0 ds_trap_original = 0 ds_no_memory = 0 ds_fragment_queue_fail = 0 ds_no_frag_entry = 0 ds_icmp_error = 0 ds_clone_fail = 0 ds_drop_pkts = 0 ], ] ] ] 2025-09-01 Mon 02:04:24:412.145 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011135616512, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: VrouterStats 2025-09-01 Mon 02:04:24:412.170 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011135616512, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:24:412.263 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011135616512, Pid 113882]: SANDESH: Send FAILED: 1756692264411518 [SYS_NOTICE]: VrouterControlStatsTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3993-1 raw_xmpp_stats= [ [ _iter306->first = 10.20.0.14 [ reconnects = 0 in_msgs = 0 out_msgs = 0 ], ] ] raw_ifmap_stats= [ [ _iter307->first = link_delete_parse_errors _iter307->second = 0, _iter307->first = link_update_parse_errors _iter307->second = 0, _iter307->first = node_delete_parse_errors _iter307->second = 0, _iter307->first = node_update_parse_errors _iter307->second = 0, ] ] raw_rt_table_size= [ [ _iter308->first = default-domain:default-project:ip-fabric:__default__ [ inet4_unicast = 4 inet4_multicast = 0 evpn = 0 bridge = 2 inet6_unicast = 2 ], _iter308->first = default-domain:default-project:ip-fabric:ip-fabric [ inet4_unicast = 2 inet4_multicast = 0 evpn = 1 bridge = 2 inet6_unicast = 0 ], ] ] ] 2025-09-01 Mon 02:04:24:412.286 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011135616512, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: VrouterControlStatsTrace 2025-09-01 Mon 02:04:24:877.779 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011152410112, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:24:877.895 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011152410112, Pid 113882]: SANDESH: Send FAILED: 1756692264877719 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.14 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:24:877.920 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011152410112, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:24:878.042 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011171994240, Pid 113882]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.14 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:04:24:878.053 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011139814912, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:24:878.177 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011139814912, Pid 113882]: SANDESH: Send FAILED: 1756692264878003 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.14 state = Connect event = Connect Event local_port = 37175 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:24:878.212 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011152410112, Pid 113882]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:04:24:878.215 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011139814912, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:24:878.405 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011139814912, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:24:878.454 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011139814912, Pid 113882]: SANDESH: Send FAILED: 1756692264878318 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.14 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:24:878.507 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011139814912, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:24:878.531 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011139814912, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:24:878.561 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011139814912, Pid 113882]: SANDESH: Send FAILED: 1756692264878334 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.14 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:24:878.580 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011139814912, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:25:011.981 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011152410112, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:25:012.079 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011152410112, Pid 113882]: SANDESH: Send FAILED: 1756692265011920 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.14 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:25:012.098 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011152410112, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:25:012.377 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011171994240, Pid 113882]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.14 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:04:25:012.381 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:25:012.574 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011160806912, Pid 113882]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:04:25:012.635 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: SANDESH: Send FAILED: 1756692265012305 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.14 state = Connect event = Connect Event local_port = 39939 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:25:012.665 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:25:012.694 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:25:012.721 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: SANDESH: Send FAILED: 1756692265012648 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.14 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:25:012.740 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:25:012.759 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:25:013.404 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: SANDESH: Send FAILED: 1756692265012657 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.14 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:25:013.430 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:25:412.594 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011156608512, Pid 113882]: Processing scm::EvIdleHoldTimerExpired in state Idle 2025-09-01 Mon 02:04:25:412.867 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011156608512, Pid 113882]: Connect : Start Connect timer 10.0.0.56:8086 2025-09-01 Mon 02:04:25:412.938 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011171994240, Pid 113882]: -(-1) OnSessionEvent TCP Connect Failed 2025-09-01 Mon 02:04:25:412.989 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011156608512, Pid 113882]: Processing scm::EvSandeshSend in state Connect 2025-09-01 Mon 02:04:25:413.022 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011171994240, Pid 113882]: -(-1):EvTcpConnectFail 2025-09-01 Mon 02:04:25:413.064 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011156608512, Pid 113882]: SANDESH: Send FAILED: 1756692265412681 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-3993-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.14 server_addrs= [ [ (*_iter6) = 10.20.0.14:5269, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.14 server_addrs= [ [ (*_iter6) = 10.20.0.14:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.56:8086, ] ] status = Initializing description = Idle : EvIdleHoldTimerExpired -> Connect ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.14, XMPP:dns-server:10.20.0.14, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:04:25:413.094 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011156608512, Pid 113882]: Wrong state: Connect for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:04:25:413.108 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011156608512, Pid 113882]: Processing scm::EvSandeshSend in state Connect 2025-09-01 Mon 02:04:25:413.270 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011156608512, Pid 113882]: SANDESH: Send FAILED: 1756692265412971 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3993-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Connect successful_connections = 0 pid = 113882 http_port = 8085 start_time = 1756692234402667 collector_name = collector_ip = 10.0.0.56:8086 collector_list= [ [ (*_iter40) = 10.0.0.56:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 15 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 14 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 19 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterControlStatsTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 44 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 12 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 34, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 80, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:04:25:413.311 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011156608512, Pid 113882]: Wrong state: Connect for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:04:25:413.324 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011156608512, Pid 113882]: Processing scm::EvTcpConnectFail in state Connect 2025-09-01 Mon 02:04:25:413.331 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011156608512, Pid 113882]: Connect : EvTcpConnectFail 2025-09-01 Mon 02:04:25:413.379 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011156608512, Pid 113882]: TCP [SYS_ERR]: TcpSessionMessageLog: Session -(-1) > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:04:25:413.411 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011156608512, Pid 113882]: Idle 2025-09-01 Mon 02:04:25:413.470 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011156608512, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:25:413.489 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011156608512, Pid 113882]: SANDESH: Send FAILED: 1756692265413354 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-3993-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.14 server_addrs= [ [ (*_iter6) = 10.20.0.14:5269, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.14 server_addrs= [ [ (*_iter6) = 10.20.0.14:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.14, XMPP:dns-server:10.20.0.14, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:04:25:413.502 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011156608512, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:04:25:413.510 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011156608512, Pid 113882]: Processing scm::EvTcpDeleteSession in state Idle 2025-09-01 Mon 02:04:25:413.553 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011156608512, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:25:413.692 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011156608512, Pid 113882]: SANDESH: Send FAILED: 1756692265413457 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3993-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Idle successful_connections = 0 pid = 113882 http_port = 8085 start_time = 1756692234402667 collector_name = collector_ip = 10.0.0.56:8086 collector_list= [ [ (*_iter40) = 10.0.0.56:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 16 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 15 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 20 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterControlStatsTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 44 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 12 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 35, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 82, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:04:25:413.716 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011156608512, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:04:30:470.808 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011171994240, Pid 113882]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.20.0.14:8085::10.20.0.14:50990 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 255 2025-09-01 Mon 02:04:30:487.878 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011171994240, Pid 113882]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.20.0.14:8085::10.20.0.14:50998 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 255 2025-09-01 Mon 02:04:33:511.939 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011171994240, Pid 113882]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.0.0.56:8085::10.0.0.56:51378 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 255 2025-09-01 Mon 02:04:46:793.089 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011171994240, Pid 113882]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.0.0.56:8085::10.0.0.56:34300 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 255 2025-09-01 Mon 02:04:52:552.980 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011135616512, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:52:553.066 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011135616512, Pid 113882]: SANDESH: Send FAILED: 1756692292552937 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.14 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:52:553.082 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011135616512, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:52:553.342 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011148211712, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:52:553.445 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011148211712, Pid 113882]: SANDESH: Send FAILED: 1756692292553268 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.14 state = Connect event = Connect Event local_port = 45345 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:52:553.472 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011148211712, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:52:553.553 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011171994240, Pid 113882]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.14 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:04:52:553.825 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:04:52:554.041 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011148211712, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:52:554.085 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011148211712, Pid 113882]: SANDESH: Send FAILED: 1756692292553952 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.14 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:52:554.105 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011148211712, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:52:554.127 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011148211712, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:52:554.154 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011148211712, Pid 113882]: SANDESH: Send FAILED: 1756692292554018 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.14 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:52:554.171 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011148211712, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:54:406.726 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011148211712, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:54:406.899 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011148211712, Pid 113882]: SANDESH: Send FAILED: 1756692294406396 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3993-1:Compute:contrail-vrouter-agent:0 sm_queue_count = 0 sm_stats= [ ev_stats= [ [ [ event = scm::EvCollectorUpdate enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvIdleHoldTimerExpired enqueues = 5 dequeues = 5 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvSandeshSend enqueues = 88 dequeues = 88 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvStart enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpConnectFail enqueues = 6 dequeues = 6 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpDeleteSession enqueues = 6 dequeues = 6 enqueue_fails = 0 dequeue_fails = 0 ], [ event = enqueues = 107 dequeues = 107 enqueue_fails = 0 dequeue_fails = 0 ], ] ] state = Idle state_since = 1756692265413410 last_state = Idle last_event = scm::EvSandeshSend last_event_at = 1756692292554124 ] max_sm_queue_count = 5 ] 2025-09-01 Mon 02:04:54:406.938 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011148211712, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:04:54:406.964 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011148211712, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:54:407.379 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011148211712, Pid 113882]: SANDESH: Send FAILED: 1756692294406599 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3993-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Idle successful_connections = 0 pid = 113882 http_port = 8085 start_time = 1756692234402667 collector_name = collector_ip = 10.0.0.56:8086 collector_list= [ [ (*_iter40) = 10.0.0.56:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 17 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 16 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 25 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterControlStatsTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 48 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 13 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 41, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 88, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:04:54:407.439 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011148211712, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:04:54:413.568 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011152410112, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:54:413.728 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011152410112, Pid 113882]: SANDESH: Send FAILED: 1756692294413436 [SYS_NOTICE]: VrouterStats: data= [ name = cn-jenkins-deploy-platform-ansible-os-3993-1 cpu_info= [ num_cpu = 8 sys_mem_info= [ total = 32 used = 0 free = 0 buffers = 0 cached = 3882065592 node_type = ] meminfo= [ virt = 32598 peakvirt = 0 res = 0 ] cpuload= [ one_min_avg = 0.2025 five_min_avg = 0.315 fifteen_min_avg = 0.26375 ] cpu_share = 0.506897 ] ] 2025-09-01 Mon 02:04:54:413.757 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011152410112, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: VrouterStats 2025-09-01 Mon 02:04:54:413.784 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011152410112, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:54:413.830 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011152410112, Pid 113882]: SANDESH: Send FAILED: 1756692294413519 [SYS_NOTICE]: ComputeCpuStateTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3993-1 cpu_info= [ [ [ mem_virt = 32598 cpu_share = 0.506897 used_sys_mem = 412901704 one_min_cpuload = 0.2025 mem_res = 0 ], ] ] ] 2025-09-01 Mon 02:04:54:413.852 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011152410112, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: ComputeCpuStateTrace 2025-09-01 Mon 02:04:54:413.880 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011152410112, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:54:414.242 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011152410112, Pid 113882]: SANDESH: Send FAILED: 1756692294413581 [SYS_NOTICE]: VrouterStats: data= [ name = cn-jenkins-deploy-platform-ansible-os-3993-1 in_tpkts = 36 in_bytes = 1624 out_tpkts = 35 out_bytes = 1554 exception_packets = 9 exception_packets_dropped = 9 raw_vhost_stats= [ name = vhost0 in_pkts = 36 in_bytes = 1624 out_pkts = 35 out_bytes = 1554 speed = -1 duplexity = -1 ] raw_vhost_drop_stats= [ ds_discard = 0 ds_pull = 0 ds_invalid_if = 0 ds_invalid_arp = 0 ds_trap_no_if = 0 ds_nowhere_to_go = 0 ds_flow_queue_limit_exceeded = 0 ds_flow_no_memory = 0 ds_flow_invalid_protocol = 0 ds_flow_nat_no_rflow = 0 ds_flow_action_drop = 0 ds_flow_action_invalid = 0 ds_flow_unusable = 0 ds_flow_table_full = 0 ds_interface_tx_discard = 0 ds_interface_drop = 0 ds_duplicated = 0 ds_push = 0 ds_ttl_exceeded = 0 ds_invalid_nh = 0 ds_invalid_label = 0 ds_invalid_protocol = 0 ds_interface_rx_discard = 0 ds_invalid_mcast_source = 0 ds_head_alloc_fail = 0 ds_pcow_fail = 0 ds_mcast_clone_fail = 0 ds_rewrite_fail = 0 ds_misc = 0 ds_invalid_packet = 0 ds_cksum_err = 0 ds_no_fmd = 0 ds_invalid_vnid = 0 ds_frag_err = 0 ds_invalid_source = 0 ds_mcast_df_bit = 0 ds_l2_no_route = 0 ds_vlan_fwd_tx = 0 ds_vlan_fwd_enq = 0 ds_drop_new_flow = 0 ds_flow_evict = 0 ds_trap_original = 0 ds_no_memory = 0 ds_fragment_queue_fail = 0 ds_no_frag_entry = 0 ds_icmp_error = 0 ds_clone_fail = 0 ds_drop_pkts = 0 ] raw_drop_stats= [ ds_discard = 0 ds_pull = 0 ds_invalid_if = 0 ds_invalid_arp = 0 ds_trap_no_if = 0 ds_nowhere_to_go = 0 ds_flow_queue_limit_exceeded = 0 ds_flow_no_memory = 0 ds_flow_invalid_protocol = 0 ds_flow_nat_no_rflow = 0 ds_flow_action_drop = 0 ds_flow_action_invalid = 0 ds_flow_unusable = 0 ds_flow_table_full = 0 ds_interface_tx_discard = 0 ds_interface_drop = 0 ds_duplicated = 0 ds_push = 0 ds_ttl_exceeded = 0 ds_invalid_nh = 0 ds_invalid_label = 0 ds_invalid_protocol = 0 ds_interface_rx_discard = 0 ds_invalid_mcast_source = 0 ds_head_alloc_fail = 0 ds_pcow_fail = 0 ds_mcast_clone_fail = 0 ds_rewrite_fail = 0 ds_misc = 0 ds_invalid_packet = 0 ds_cksum_err = 0 ds_no_fmd = 0 ds_invalid_vnid = 0 ds_frag_err = 0 ds_invalid_source = 0 ds_mcast_df_bit = 0 ds_l2_no_route = 0 ds_vlan_fwd_tx = 0 ds_vlan_fwd_enq = 0 ds_drop_new_flow = 0 ds_flow_evict = 0 ds_trap_original = 0 ds_no_memory = 0 ds_fragment_queue_fail = 0 ds_no_frag_entry = 0 ds_icmp_error = 0 ds_clone_fail = 0 ds_drop_pkts = 0 ] flow_rate= [ added_flows = 0 max_flow_adds_per_second = 0 min_flow_adds_per_second = 0 deleted_flows = 0 max_flow_deletes_per_second = 0 min_flow_deletes_per_second = 0 active_flows = 0 hold_flows = 0 ] raw_phy_if_stats= [ [ _iter262->first = ens6 [ in_pkts = 3 in_bytes = 210 out_pkts = 36 out_bytes = 1624 ], ] ] raw_phy_if_drop_stats= [ [ _iter263->first = ens6 [ ds_discard = 0 ds_pull = 0 ds_invalid_if = 0 ds_invalid_arp = 0 ds_trap_no_if = 0 ds_nowhere_to_go = 0 ds_flow_queue_limit_exceeded = 0 ds_flow_no_memory = 0 ds_flow_invalid_protocol = 0 ds_flow_nat_no_rflow = 0 ds_flow_action_drop = 0 ds_flow_action_invalid = 0 ds_flow_unusable = 0 ds_flow_table_full = 0 ds_interface_tx_discard = 0 ds_interface_drop = 0 ds_duplicated = 0 ds_push = 0 ds_ttl_exceeded = 0 ds_invalid_nh = 0 ds_invalid_label = 0 ds_invalid_protocol = 0 ds_interface_rx_discard = 0 ds_invalid_mcast_source = 0 ds_head_alloc_fail = 0 ds_pcow_fail = 0 ds_mcast_clone_fail = 0 ds_rewrite_fail = 0 ds_misc = 0 ds_invalid_packet = 0 ds_cksum_err = 0 ds_no_fmd = 0 ds_invalid_vnid = 0 ds_frag_err = 0 ds_invalid_source = 0 ds_mcast_df_bit = 0 ds_l2_no_route = 0 ds_vlan_fwd_tx = 0 ds_vlan_fwd_enq = 0 ds_drop_new_flow = 0 ds_flow_evict = 0 ds_trap_original = 0 ds_no_memory = 0 ds_fragment_queue_fail = 0 ds_no_frag_entry = 0 ds_icmp_error = 0 ds_clone_fail = 0 ds_drop_pkts = 0 ], ] ] ] 2025-09-01 Mon 02:04:54:414.299 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011152410112, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: VrouterStats 2025-09-01 Mon 02:04:54:414.325 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011152410112, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:54:414.399 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011152410112, Pid 113882]: SANDESH: Send FAILED: 1756692294413652 [SYS_NOTICE]: VrouterControlStatsTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3993-1 raw_xmpp_stats= [ [ _iter306->first = 10.20.0.14 [ reconnects = 0 in_msgs = 0 out_msgs = 0 ], ] ] raw_ifmap_stats= [ [ _iter307->first = link_delete_parse_errors _iter307->second = 0, _iter307->first = link_update_parse_errors _iter307->second = 0, _iter307->first = node_delete_parse_errors _iter307->second = 0, _iter307->first = node_update_parse_errors _iter307->second = 0, ] ] raw_rt_table_size= [ [ _iter308->first = default-domain:default-project:ip-fabric:__default__ [ inet4_unicast = 4 inet4_multicast = 0 evpn = 0 bridge = 2 inet6_unicast = 2 ], _iter308->first = default-domain:default-project:ip-fabric:ip-fabric [ inet4_unicast = 2 inet4_multicast = 0 evpn = 1 bridge = 2 inet6_unicast = 0 ], ] ] ] 2025-09-01 Mon 02:04:54:414.421 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011152410112, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: VrouterControlStatsTrace 2025-09-01 Mon 02:04:55:413.643 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011156608512, Pid 113882]: Processing scm::EvIdleHoldTimerExpired in state Idle 2025-09-01 Mon 02:04:55:414.082 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011156608512, Pid 113882]: Connect : Start Connect timer 10.0.0.56:8086 2025-09-01 Mon 02:04:55:414.173 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011171994240, Pid 113882]: -(-1) OnSessionEvent TCP Connect Failed 2025-09-01 Mon 02:04:55:414.280 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011171994240, Pid 113882]: -(-1):EvTcpConnectFail 2025-09-01 Mon 02:04:55:414.283 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011156608512, Pid 113882]: Processing scm::EvSandeshSend in state Connect 2025-09-01 Mon 02:04:55:414.409 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011156608512, Pid 113882]: SANDESH: Send FAILED: 1756692295413777 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-3993-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.14 server_addrs= [ [ (*_iter6) = 10.20.0.14:5269, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.14 server_addrs= [ [ (*_iter6) = 10.20.0.14:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.56:8086, ] ] status = Initializing description = Idle : EvIdleHoldTimerExpired -> Connect ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.14, XMPP:dns-server:10.20.0.14, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:04:55:414.444 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011156608512, Pid 113882]: Wrong state: Connect for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:04:55:414.468 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011156608512, Pid 113882]: Processing scm::EvSandeshSend in state Connect 2025-09-01 Mon 02:04:55:414.865 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011156608512, Pid 113882]: SANDESH: Send FAILED: 1756692295414250 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3993-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Connect successful_connections = 0 pid = 113882 http_port = 8085 start_time = 1756692234402667 collector_name = collector_ip = 10.0.0.56:8086 collector_list= [ [ (*_iter40) = 10.0.0.56:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ComputeCpuStateTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 17 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 18 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 25 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterControlStatsTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 3 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 48 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 13 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 41, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 94, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:04:55:414.915 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011156608512, Pid 113882]: Wrong state: Connect for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:04:55:414.945 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011156608512, Pid 113882]: Processing scm::EvTcpConnectFail in state Connect 2025-09-01 Mon 02:04:55:414.961 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011156608512, Pid 113882]: Connect : EvTcpConnectFail 2025-09-01 Mon 02:04:55:415.048 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011156608512, Pid 113882]: TCP [SYS_ERR]: TcpSessionMessageLog: Session -(-1) > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:04:55:415.107 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011156608512, Pid 113882]: Idle 2025-09-01 Mon 02:04:55:415.244 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011156608512, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:55:415.291 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011156608512, Pid 113882]: SANDESH: Send FAILED: 1756692295415003 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-3993-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.14 server_addrs= [ [ (*_iter6) = 10.20.0.14:5269, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.14 server_addrs= [ [ (*_iter6) = 10.20.0.14:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.14, XMPP:dns-server:10.20.0.14, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:04:55:415.306 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011156608512, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:04:55:415.337 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011156608512, Pid 113882]: Processing scm::EvTcpDeleteSession in state Idle 2025-09-01 Mon 02:04:55:415.391 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011156608512, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:55:415.720 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011156608512, Pid 113882]: SANDESH: Send FAILED: 1756692295415218 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3993-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Idle successful_connections = 0 pid = 113882 http_port = 8085 start_time = 1756692234402667 collector_name = collector_ip = 10.0.0.56:8086 collector_list= [ [ (*_iter40) = 10.0.0.56:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ComputeCpuStateTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 18 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 19 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 26 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterControlStatsTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 3 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 48 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 13 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 42, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 96, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:04:55:415.767 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011156608512, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:04:56:198.634 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011152410112, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:56:198.746 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011152410112, Pid 113882]: SANDESH: Send FAILED: 1756692296198581 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.14 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:56:198.776 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011152410112, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:56:198.817 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011152410112, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:56:198.860 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011152410112, Pid 113882]: SANDESH: Send FAILED: 1756692296198802 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.14 state = Connect event = Connect Event local_port = 44919 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:56:198.913 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011171994240, Pid 113882]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.14 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:04:56:198.936 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011152410112, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:56:199.115 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011152410112, Pid 113882]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:04:56:199.248 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011139814912, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:56:199.341 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011139814912, Pid 113882]: SANDESH: Send FAILED: 1756692296199181 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.14 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:56:199.370 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011139814912, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:56:199.396 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011139814912, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:56:199.453 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011139814912, Pid 113882]: SANDESH: Send FAILED: 1756692296199212 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.14 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:56:199.472 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011139814912, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:59:501.941 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011171994240, Pid 113882]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.0.0.56:8085::10.0.0.56:56210 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 255 2025-09-01 Mon 02:05:12:790.992 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011171994240, Pid 113882]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.0.0.56:8085::10.0.0.56:43474 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 255 2025-09-01 Mon 02:05:22:794.352 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:22:794.474 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: SANDESH: Send FAILED: 1756692322794291 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.14 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:22:794.502 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:22:794.529 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:22:794.564 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: SANDESH: Send FAILED: 1756692322794493 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.14 state = Connect event = Connect Event local_port = 32859 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:22:794.583 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:22:794.606 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011171994240, Pid 113882]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.14 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:05:22:794.788 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011148211712, Pid 113882]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:05:22:794.857 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:22:794.892 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: SANDESH: Send FAILED: 1756692322794839 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.14 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:22:794.911 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:22:794.932 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:22:794.981 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: SANDESH: Send FAILED: 1756692322794847 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.14 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:22:795.000 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:24:407.020 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:24:407.121 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: SANDESH: Send FAILED: 1756692324406862 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3993-1:Compute:contrail-vrouter-agent:0 sm_queue_count = 0 sm_stats= [ ev_stats= [ [ [ event = scm::EvCollectorUpdate enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvIdleHoldTimerExpired enqueues = 6 dequeues = 6 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvSandeshSend enqueues = 106 dequeues = 106 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvStart enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpConnectFail enqueues = 7 dequeues = 7 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpDeleteSession enqueues = 7 dequeues = 7 enqueue_fails = 0 dequeue_fails = 0 ], [ event = enqueues = 128 dequeues = 128 enqueue_fails = 0 dequeue_fails = 0 ], ] ] state = Idle state_since = 1756692295415105 last_state = Idle last_event = scm::EvSandeshSend last_event_at = 1756692322794929 ] max_sm_queue_count = 5 ] 2025-09-01 Mon 02:05:24:407.140 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:05:24:407.152 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:24:407.432 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: SANDESH: Send FAILED: 1756692324406982 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3993-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Idle successful_connections = 0 pid = 113882 http_port = 8085 start_time = 1756692234402667 collector_name = collector_ip = 10.0.0.56:8086 collector_list= [ [ (*_iter40) = 10.0.0.56:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ComputeCpuStateTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 19 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 20 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 30 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterControlStatsTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 3 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 56 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 15 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 48, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 106, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:05:24:407.467 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:05:24:414.042 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011156608512, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:24:414.560 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011156608512, Pid 113882]: SANDESH: Send FAILED: 1756692324413975 [SYS_NOTICE]: VrouterStats: data= [ name = cn-jenkins-deploy-platform-ansible-os-3993-1 in_tpkts = 51 in_bytes = 2254 out_tpkts = 50 out_bytes = 2184 exception_packets = 10 exception_packets_dropped = 10 raw_vhost_stats= [ name = vhost0 in_pkts = 51 in_bytes = 2254 out_pkts = 50 out_bytes = 2184 speed = -1 duplexity = -1 ] raw_vhost_drop_stats= [ ds_discard = 0 ds_pull = 0 ds_invalid_if = 0 ds_invalid_arp = 0 ds_trap_no_if = 0 ds_nowhere_to_go = 0 ds_flow_queue_limit_exceeded = 0 ds_flow_no_memory = 0 ds_flow_invalid_protocol = 0 ds_flow_nat_no_rflow = 0 ds_flow_action_drop = 0 ds_flow_action_invalid = 0 ds_flow_unusable = 0 ds_flow_table_full = 0 ds_interface_tx_discard = 0 ds_interface_drop = 0 ds_duplicated = 0 ds_push = 0 ds_ttl_exceeded = 0 ds_invalid_nh = 0 ds_invalid_label = 0 ds_invalid_protocol = 0 ds_interface_rx_discard = 0 ds_invalid_mcast_source = 0 ds_head_alloc_fail = 0 ds_pcow_fail = 0 ds_mcast_clone_fail = 0 ds_rewrite_fail = 0 ds_misc = 0 ds_invalid_packet = 0 ds_cksum_err = 0 ds_no_fmd = 0 ds_invalid_vnid = 0 ds_frag_err = 0 ds_invalid_source = 0 ds_mcast_df_bit = 0 ds_l2_no_route = 0 ds_vlan_fwd_tx = 0 ds_vlan_fwd_enq = 0 ds_drop_new_flow = 0 ds_flow_evict = 0 ds_trap_original = 0 ds_no_memory = 0 ds_fragment_queue_fail = 0 ds_no_frag_entry = 0 ds_icmp_error = 0 ds_clone_fail = 0 ds_drop_pkts = 0 ] raw_drop_stats= [ ds_discard = 0 ds_pull = 0 ds_invalid_if = 0 ds_invalid_arp = 0 ds_trap_no_if = 0 ds_nowhere_to_go = 0 ds_flow_queue_limit_exceeded = 0 ds_flow_no_memory = 0 ds_flow_invalid_protocol = 0 ds_flow_nat_no_rflow = 0 ds_flow_action_drop = 0 ds_flow_action_invalid = 0 ds_flow_unusable = 0 ds_flow_table_full = 0 ds_interface_tx_discard = 0 ds_interface_drop = 0 ds_duplicated = 0 ds_push = 0 ds_ttl_exceeded = 0 ds_invalid_nh = 0 ds_invalid_label = 0 ds_invalid_protocol = 0 ds_interface_rx_discard = 0 ds_invalid_mcast_source = 0 ds_head_alloc_fail = 0 ds_pcow_fail = 0 ds_mcast_clone_fail = 0 ds_rewrite_fail = 0 ds_misc = 0 ds_invalid_packet = 0 ds_cksum_err = 0 ds_no_fmd = 0 ds_invalid_vnid = 0 ds_frag_err = 0 ds_invalid_source = 0 ds_mcast_df_bit = 0 ds_l2_no_route = 0 ds_vlan_fwd_tx = 0 ds_vlan_fwd_enq = 0 ds_drop_new_flow = 0 ds_flow_evict = 0 ds_trap_original = 0 ds_no_memory = 0 ds_fragment_queue_fail = 0 ds_no_frag_entry = 0 ds_icmp_error = 0 ds_clone_fail = 0 ds_drop_pkts = 0 ] total_in_bandwidth_utilization = 4.87891e-17 total_out_bandwidth_utilization = 9.59519e-16 flow_rate= [ added_flows = 0 max_flow_adds_per_second = 0 min_flow_adds_per_second = 0 deleted_flows = 0 max_flow_deletes_per_second = 0 min_flow_deletes_per_second = 0 active_flows = 0 hold_flows = 0 ] phy_band_in_bps= [ [ _iter260->first = ens6 _iter260->second = 9, ] ] phy_band_out_bps= [ [ _iter261->first = ens6 _iter261->second = 177, ] ] raw_phy_if_stats= [ [ _iter262->first = ens6 [ in_pkts = 3 in_bytes = 210 out_pkts = 51 out_bytes = 2254 ], ] ] raw_phy_if_drop_stats= [ [ _iter263->first = ens6 [ ds_discard = 0 ds_pull = 0 ds_invalid_if = 0 ds_invalid_arp = 0 ds_trap_no_if = 0 ds_nowhere_to_go = 0 ds_flow_queue_limit_exceeded = 0 ds_flow_no_memory = 0 ds_flow_invalid_protocol = 0 ds_flow_nat_no_rflow = 0 ds_flow_action_drop = 0 ds_flow_action_invalid = 0 ds_flow_unusable = 0 ds_flow_table_full = 0 ds_interface_tx_discard = 0 ds_interface_drop = 0 ds_duplicated = 0 ds_push = 0 ds_ttl_exceeded = 0 ds_invalid_nh = 0 ds_invalid_label = 0 ds_invalid_protocol = 0 ds_interface_rx_discard = 0 ds_invalid_mcast_source = 0 ds_head_alloc_fail = 0 ds_pcow_fail = 0 ds_mcast_clone_fail = 0 ds_rewrite_fail = 0 ds_misc = 0 ds_invalid_packet = 0 ds_cksum_err = 0 ds_no_fmd = 0 ds_invalid_vnid = 0 ds_frag_err = 0 ds_invalid_source = 0 ds_mcast_df_bit = 0 ds_l2_no_route = 0 ds_vlan_fwd_tx = 0 ds_vlan_fwd_enq = 0 ds_drop_new_flow = 0 ds_flow_evict = 0 ds_trap_original = 0 ds_no_memory = 0 ds_fragment_queue_fail = 0 ds_no_frag_entry = 0 ds_icmp_error = 0 ds_clone_fail = 0 ds_drop_pkts = 0 ], ] ] ] 2025-09-01 Mon 02:05:24:414.633 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011156608512, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: VrouterStats 2025-09-01 Mon 02:05:24:414.665 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011156608512, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:24:414.784 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011156608512, Pid 113882]: SANDESH: Send FAILED: 1756692324414047 [SYS_NOTICE]: VrouterControlStatsTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3993-1 raw_xmpp_stats= [ [ _iter306->first = 10.20.0.14 [ reconnects = 0 in_msgs = 0 out_msgs = 0 ], ] ] raw_ifmap_stats= [ [ _iter307->first = link_delete_parse_errors _iter307->second = 0, _iter307->first = link_update_parse_errors _iter307->second = 0, _iter307->first = node_delete_parse_errors _iter307->second = 0, _iter307->first = node_update_parse_errors _iter307->second = 0, ] ] raw_rt_table_size= [ [ _iter308->first = default-domain:default-project:ip-fabric:__default__ [ inet4_unicast = 4 inet4_multicast = 0 evpn = 0 bridge = 2 inet6_unicast = 2 ], _iter308->first = default-domain:default-project:ip-fabric:ip-fabric [ inet4_unicast = 2 inet4_multicast = 0 evpn = 1 bridge = 2 inet6_unicast = 0 ], ] ] ] 2025-09-01 Mon 02:05:24:414.816 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011156608512, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: VrouterControlStatsTrace 2025-09-01 Mon 02:05:25:415.309 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011135616512, Pid 113882]: Processing scm::EvIdleHoldTimerExpired in state Idle 2025-09-01 Mon 02:05:25:415.666 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011135616512, Pid 113882]: Connect : Start Connect timer 10.0.0.56:8086 2025-09-01 Mon 02:05:25:415.706 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011171994240, Pid 113882]: -(-1) OnSessionEvent TCP Connect Failed 2025-09-01 Mon 02:05:25:415.756 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011171994240, Pid 113882]: -(-1):EvTcpConnectFail 2025-09-01 Mon 02:05:25:415.858 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011135616512, Pid 113882]: Processing scm::EvSandeshSend in state Connect 2025-09-01 Mon 02:05:25:415.900 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011135616512, Pid 113882]: SANDESH: Send FAILED: 1756692325415413 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-3993-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.14 server_addrs= [ [ (*_iter6) = 10.20.0.14:5269, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.14 server_addrs= [ [ (*_iter6) = 10.20.0.14:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.56:8086, ] ] status = Initializing description = Idle : EvIdleHoldTimerExpired -> Connect ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.14, XMPP:dns-server:10.20.0.14, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:05:25:415.915 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011135616512, Pid 113882]: Wrong state: Connect for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:05:25:415.929 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011135616512, Pid 113882]: Processing scm::EvTcpConnectFail in state Connect 2025-09-01 Mon 02:05:25:415.940 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011135616512, Pid 113882]: Connect : EvTcpConnectFail 2025-09-01 Mon 02:05:25:415.999 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011135616512, Pid 113882]: TCP [SYS_ERR]: TcpSessionMessageLog: Session -(-1) > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:05:25:416.044 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011135616512, Pid 113882]: Idle 2025-09-01 Mon 02:05:25:416.122 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011135616512, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:25:416.453 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011135616512, Pid 113882]: SANDESH: Send FAILED: 1756692325415833 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3993-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Connect successful_connections = 0 pid = 113882 http_port = 8085 start_time = 1756692234402667 collector_name = collector_ip = 10.0.0.56:8086 collector_list= [ [ (*_iter40) = 10.0.0.56:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ComputeCpuStateTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 19 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 22 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 30 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterControlStatsTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 3 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 4 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 56 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 15 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 48, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 110, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:05:25:416.500 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011135616512, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:05:25:416.517 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011135616512, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:25:416.550 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011135616512, Pid 113882]: SANDESH: Send FAILED: 1756692325415971 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-3993-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.14 server_addrs= [ [ (*_iter6) = 10.20.0.14:5269, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.14 server_addrs= [ [ (*_iter6) = 10.20.0.14:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.14, XMPP:dns-server:10.20.0.14, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:05:25:416.562 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011135616512, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:05:25:416.578 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011135616512, Pid 113882]: Processing scm::EvTcpDeleteSession in state Idle 2025-09-01 Mon 02:05:25:416.621 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011135616512, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:25:416.837 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011135616512, Pid 113882]: SANDESH: Send FAILED: 1756692325416105 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3993-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Idle successful_connections = 0 pid = 113882 http_port = 8085 start_time = 1756692234402667 collector_name = collector_ip = 10.0.0.56:8086 collector_list= [ [ (*_iter40) = 10.0.0.56:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ComputeCpuStateTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 20 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 22 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 31 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterControlStatsTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 3 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 4 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 56 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 15 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 49, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 111, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:05:25:416.867 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011135616512, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:05:25:874.120 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011171994240, Pid 113882]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.0.0.56:8085::10.0.0.56:46320 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 255 2025-09-01 Mon 02:05:25:899.595 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:25:899.725 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: SANDESH: Send FAILED: 1756692325899524 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.14 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:25:899.760 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:25:899.939 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011148211712, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:25:899.940 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011171994240, Pid 113882]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.14 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:05:25:900.049 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011148211712, Pid 113882]: SANDESH: Send FAILED: 1756692325899872 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.14 state = Connect event = Connect Event local_port = 44969 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:25:900.258 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011160806912, Pid 113882]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:05:25:900.288 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011148211712, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:25:900.466 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011148211712, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:25:900.508 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011148211712, Pid 113882]: SANDESH: Send FAILED: 1756692325900384 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.14 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:25:900.528 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011148211712, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:25:900.548 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011148211712, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:25:900.576 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011148211712, Pid 113882]: SANDESH: Send FAILED: 1756692325900399 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.14 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:25:900.599 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011148211712, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:31:459.649 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011171994240, Pid 113882]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.20.0.14:8085::10.20.0.14:55832 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 255 2025-09-01 Mon 02:05:31:467.946 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011171994240, Pid 113882]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.20.0.14:8085::10.20.0.14:55840 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 255 2025-09-01 Mon 02:05:38:710.701 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011171994240, Pid 113882]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.0.0.56:8085::10.0.0.56:43126 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 255 2025-09-01 Mon 02:05:51:568.935 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011171994240, Pid 113882]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.0.0.56:8085::10.0.0.56:50294 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 255 2025-09-01 Mon 02:05:53:980.868 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011135616512, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:53:980.995 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011135616512, Pid 113882]: SANDESH: Send FAILED: 1756692353980803 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.14 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:53:981.050 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011135616512, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:53:981.276 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:53:981.320 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011171994240, Pid 113882]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.14 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:05:53:981.422 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: SANDESH: Send FAILED: 1756692353981174 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.14 state = Connect event = Connect Event local_port = 44551 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:53:981.532 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:53:981.603 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011148211712, Pid 113882]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:05:53:981.855 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011156608512, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:53:981.981 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011156608512, Pid 113882]: SANDESH: Send FAILED: 1756692353981709 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.14 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:53:982.026 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011156608512, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:53:982.067 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011156608512, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:53:982.122 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011156608512, Pid 113882]: SANDESH: Send FAILED: 1756692353981759 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.14 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:53:982.154 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011156608512, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:54:407.627 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011156608512, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:54:407.802 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011156608512, Pid 113882]: SANDESH: Send FAILED: 1756692354407307 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3993-1:Compute:contrail-vrouter-agent:0 sm_queue_count = 0 sm_stats= [ ev_stats= [ [ [ event = scm::EvCollectorUpdate enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvIdleHoldTimerExpired enqueues = 7 dequeues = 7 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvSandeshSend enqueues = 122 dequeues = 122 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvStart enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpConnectFail enqueues = 8 dequeues = 8 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpDeleteSession enqueues = 8 dequeues = 8 enqueue_fails = 0 dequeue_fails = 0 ], [ event = enqueues = 147 dequeues = 147 enqueue_fails = 0 dequeue_fails = 0 ], ] ] state = Idle state_since = 1756692325416042 last_state = Idle last_event = scm::EvSandeshSend last_event_at = 1756692353982059 ] max_sm_queue_count = 5 ] 2025-09-01 Mon 02:05:54:407.873 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011156608512, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:05:54:407.899 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011156608512, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:54:408.364 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011156608512, Pid 113882]: SANDESH: Send FAILED: 1756692354407523 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3993-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Idle successful_connections = 0 pid = 113882 http_port = 8085 start_time = 1756692234402667 collector_name = collector_ip = 10.0.0.56:8086 collector_list= [ [ (*_iter40) = 10.0.0.56:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ComputeCpuStateTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 21 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 24 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 38 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterControlStatsTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 3 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 4 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 64 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 17 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 58, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 122, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:05:54:408.415 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011156608512, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:05:54:414.488 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011152410112, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:54:414.643 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011152410112, Pid 113882]: SANDESH: Send FAILED: 1756692354414425 [SYS_NOTICE]: VrouterStats: data= [ name = cn-jenkins-deploy-platform-ansible-os-3993-1 cpu_info= [ num_cpu = 8 sys_mem_info= [ total = 32 used = 0 free = 0 buffers = 0 cached = 3882065592 node_type = ] meminfo= [ virt = 32598 peakvirt = 0 res = 0 ] cpuload= [ one_min_avg = 0.14 five_min_avg = 0.27875 fifteen_min_avg = 0.25375 ] cpu_share = 0.460417 ] ] 2025-09-01 Mon 02:05:54:414.672 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011152410112, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: VrouterStats 2025-09-01 Mon 02:05:54:414.699 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011152410112, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:54:414.744 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011152410112, Pid 113882]: SANDESH: Send FAILED: 1756692354414468 [SYS_NOTICE]: ComputeCpuStateTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3993-1 cpu_info= [ [ [ mem_virt = 32598 cpu_share = 0.460417 used_sys_mem = 412901704 one_min_cpuload = 0.14 mem_res = 0 ], ] ] ] 2025-09-01 Mon 02:05:54:414.765 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011152410112, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: ComputeCpuStateTrace 2025-09-01 Mon 02:05:54:414.786 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011152410112, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:54:415.103 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011152410112, Pid 113882]: SANDESH: Send FAILED: 1756692354414530 [SYS_NOTICE]: VrouterStats: data= [ name = cn-jenkins-deploy-platform-ansible-os-3993-1 in_tpkts = 66 in_bytes = 2884 out_tpkts = 66 out_bytes = 2884 raw_vhost_stats= [ name = vhost0 in_pkts = 66 in_bytes = 2884 out_pkts = 66 out_bytes = 2884 speed = -1 duplexity = -1 ] raw_vhost_drop_stats= [ ds_discard = 0 ds_pull = 0 ds_invalid_if = 0 ds_invalid_arp = 0 ds_trap_no_if = 0 ds_nowhere_to_go = 0 ds_flow_queue_limit_exceeded = 0 ds_flow_no_memory = 0 ds_flow_invalid_protocol = 0 ds_flow_nat_no_rflow = 0 ds_flow_action_drop = 0 ds_flow_action_invalid = 0 ds_flow_unusable = 0 ds_flow_table_full = 0 ds_interface_tx_discard = 0 ds_interface_drop = 0 ds_duplicated = 0 ds_push = 0 ds_ttl_exceeded = 0 ds_invalid_nh = 0 ds_invalid_label = 0 ds_invalid_protocol = 0 ds_interface_rx_discard = 0 ds_invalid_mcast_source = 0 ds_head_alloc_fail = 0 ds_pcow_fail = 0 ds_mcast_clone_fail = 0 ds_rewrite_fail = 0 ds_misc = 0 ds_invalid_packet = 0 ds_cksum_err = 0 ds_no_fmd = 0 ds_invalid_vnid = 0 ds_frag_err = 0 ds_invalid_source = 0 ds_mcast_df_bit = 0 ds_l2_no_route = 0 ds_vlan_fwd_tx = 0 ds_vlan_fwd_enq = 0 ds_drop_new_flow = 0 ds_flow_evict = 0 ds_trap_original = 0 ds_no_memory = 0 ds_fragment_queue_fail = 0 ds_no_frag_entry = 0 ds_icmp_error = 0 ds_clone_fail = 0 ds_drop_pkts = 0 ] raw_drop_stats= [ ds_discard = 0 ds_pull = 0 ds_invalid_if = 0 ds_invalid_arp = 0 ds_trap_no_if = 0 ds_nowhere_to_go = 0 ds_flow_queue_limit_exceeded = 0 ds_flow_no_memory = 0 ds_flow_invalid_protocol = 0 ds_flow_nat_no_rflow = 0 ds_flow_action_drop = 0 ds_flow_action_invalid = 0 ds_flow_unusable = 0 ds_flow_table_full = 0 ds_interface_tx_discard = 0 ds_interface_drop = 0 ds_duplicated = 0 ds_push = 0 ds_ttl_exceeded = 0 ds_invalid_nh = 0 ds_invalid_label = 0 ds_invalid_protocol = 0 ds_interface_rx_discard = 0 ds_invalid_mcast_source = 0 ds_head_alloc_fail = 0 ds_pcow_fail = 0 ds_mcast_clone_fail = 0 ds_rewrite_fail = 0 ds_misc = 0 ds_invalid_packet = 0 ds_cksum_err = 0 ds_no_fmd = 0 ds_invalid_vnid = 0 ds_frag_err = 0 ds_invalid_source = 0 ds_mcast_df_bit = 0 ds_l2_no_route = 0 ds_vlan_fwd_tx = 0 ds_vlan_fwd_enq = 0 ds_drop_new_flow = 0 ds_flow_evict = 0 ds_trap_original = 0 ds_no_memory = 0 ds_fragment_queue_fail = 0 ds_no_frag_entry = 0 ds_icmp_error = 0 ds_clone_fail = 0 ds_drop_pkts = 0 ] flow_rate= [ added_flows = 0 max_flow_adds_per_second = 0 min_flow_adds_per_second = 0 deleted_flows = 0 max_flow_deletes_per_second = 0 min_flow_deletes_per_second = 0 active_flows = 0 hold_flows = 0 ] raw_phy_if_stats= [ [ _iter262->first = ens6 [ in_pkts = 4 in_bytes = 280 out_pkts = 66 out_bytes = 2884 ], ] ] raw_phy_if_drop_stats= [ [ _iter263->first = ens6 [ ds_discard = 0 ds_pull = 0 ds_invalid_if = 0 ds_invalid_arp = 0 ds_trap_no_if = 0 ds_nowhere_to_go = 0 ds_flow_queue_limit_exceeded = 0 ds_flow_no_memory = 0 ds_flow_invalid_protocol = 0 ds_flow_nat_no_rflow = 0 ds_flow_action_drop = 0 ds_flow_action_invalid = 0 ds_flow_unusable = 0 ds_flow_table_full = 0 ds_interface_tx_discard = 0 ds_interface_drop = 0 ds_duplicated = 0 ds_push = 0 ds_ttl_exceeded = 0 ds_invalid_nh = 0 ds_invalid_label = 0 ds_invalid_protocol = 0 ds_interface_rx_discard = 0 ds_invalid_mcast_source = 0 ds_head_alloc_fail = 0 ds_pcow_fail = 0 ds_mcast_clone_fail = 0 ds_rewrite_fail = 0 ds_misc = 0 ds_invalid_packet = 0 ds_cksum_err = 0 ds_no_fmd = 0 ds_invalid_vnid = 0 ds_frag_err = 0 ds_invalid_source = 0 ds_mcast_df_bit = 0 ds_l2_no_route = 0 ds_vlan_fwd_tx = 0 ds_vlan_fwd_enq = 0 ds_drop_new_flow = 0 ds_flow_evict = 0 ds_trap_original = 0 ds_no_memory = 0 ds_fragment_queue_fail = 0 ds_no_frag_entry = 0 ds_icmp_error = 0 ds_clone_fail = 0 ds_drop_pkts = 0 ], ] ] ] 2025-09-01 Mon 02:05:54:415.161 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011152410112, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: VrouterStats 2025-09-01 Mon 02:05:54:415.214 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011152410112, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:54:415.288 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011152410112, Pid 113882]: SANDESH: Send FAILED: 1756692354414621 [SYS_NOTICE]: VrouterControlStatsTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3993-1 raw_xmpp_stats= [ [ _iter306->first = 10.20.0.14 [ reconnects = 0 in_msgs = 0 out_msgs = 0 ], ] ] raw_ifmap_stats= [ [ _iter307->first = link_delete_parse_errors _iter307->second = 0, _iter307->first = link_update_parse_errors _iter307->second = 0, _iter307->first = node_delete_parse_errors _iter307->second = 0, _iter307->first = node_update_parse_errors _iter307->second = 0, ] ] raw_rt_table_size= [ [ _iter308->first = default-domain:default-project:ip-fabric:__default__ [ inet4_unicast = 4 inet4_multicast = 0 evpn = 0 bridge = 2 inet6_unicast = 2 ], _iter308->first = default-domain:default-project:ip-fabric:ip-fabric [ inet4_unicast = 2 inet4_multicast = 0 evpn = 1 bridge = 2 inet6_unicast = 0 ], ] ] ] 2025-09-01 Mon 02:05:54:415.317 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011152410112, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: VrouterControlStatsTrace 2025-09-01 Mon 02:05:54:655.170 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011156608512, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:54:655.336 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011156608512, Pid 113882]: SANDESH: Send FAILED: 1756692354655090 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.14 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:54:655.374 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011156608512, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:54:655.594 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011152410112, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:54:655.607 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011171994240, Pid 113882]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.14 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:05:54:655.710 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011152410112, Pid 113882]: SANDESH: Send FAILED: 1756692354655522 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.14 state = Connect event = Connect Event local_port = 34819 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:54:655.835 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011152410112, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:54:655.884 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011148211712, Pid 113882]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:05:54:656.114 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011135616512, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:54:656.233 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011135616512, Pid 113882]: SANDESH: Send FAILED: 1756692354655991 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.14 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:54:656.262 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011135616512, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:54:656.288 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011135616512, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:54:656.320 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011135616512, Pid 113882]: SANDESH: Send FAILED: 1756692354656039 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.14 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:54:656.337 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011135616512, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:55:416.332 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011139814912, Pid 113882]: Processing scm::EvIdleHoldTimerExpired in state Idle 2025-09-01 Mon 02:05:55:416.697 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011139814912, Pid 113882]: Connect : Start Connect timer 10.0.0.56:8086 2025-09-01 Mon 02:05:55:416.774 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011171994240, Pid 113882]: -(-1) OnSessionEvent TCP Connect Failed 2025-09-01 Mon 02:05:55:416.820 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011139814912, Pid 113882]: Processing scm::EvSandeshSend in state Connect 2025-09-01 Mon 02:05:55:416.869 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011171994240, Pid 113882]: -(-1):EvTcpConnectFail 2025-09-01 Mon 02:05:55:416.910 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011139814912, Pid 113882]: SANDESH: Send FAILED: 1756692355416448 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-3993-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.14 server_addrs= [ [ (*_iter6) = 10.20.0.14:5269, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.14 server_addrs= [ [ (*_iter6) = 10.20.0.14:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.56:8086, ] ] status = Initializing description = Idle : EvIdleHoldTimerExpired -> Connect ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.14, XMPP:dns-server:10.20.0.14, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:05:55:416.920 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011139814912, Pid 113882]: Wrong state: Connect for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:05:55:416.930 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011139814912, Pid 113882]: Processing scm::EvSandeshSend in state Connect 2025-09-01 Mon 02:05:55:417.069 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011139814912, Pid 113882]: SANDESH: Send FAILED: 1756692355416783 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3993-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Connect successful_connections = 0 pid = 113882 http_port = 8085 start_time = 1756692234402667 collector_name = collector_ip = 10.0.0.56:8086 collector_list= [ [ (*_iter40) = 10.0.0.56:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ComputeCpuStateTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 21 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 26 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 39 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterControlStatsTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 4 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 6 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 68 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 18 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 60, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 132, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:05:55:417.095 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011139814912, Pid 113882]: Wrong state: Connect for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:05:55:417.104 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011139814912, Pid 113882]: Processing scm::EvTcpConnectFail in state Connect 2025-09-01 Mon 02:05:55:417.111 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011139814912, Pid 113882]: Connect : EvTcpConnectFail 2025-09-01 Mon 02:05:55:417.153 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011139814912, Pid 113882]: TCP [SYS_ERR]: TcpSessionMessageLog: Session -(-1) > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:05:55:417.199 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011139814912, Pid 113882]: Idle 2025-09-01 Mon 02:05:55:417.272 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011139814912, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:55:417.292 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011139814912, Pid 113882]: SANDESH: Send FAILED: 1756692355417131 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-3993-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.14 server_addrs= [ [ (*_iter6) = 10.20.0.14:5269, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.14 server_addrs= [ [ (*_iter6) = 10.20.0.14:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.14, XMPP:dns-server:10.20.0.14, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:05:55:417.304 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011139814912, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:05:55:417.311 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011139814912, Pid 113882]: Processing scm::EvTcpDeleteSession in state Idle 2025-09-01 Mon 02:05:55:417.341 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011139814912, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:55:417.461 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011139814912, Pid 113882]: SANDESH: Send FAILED: 1756692355417259 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3993-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Idle successful_connections = 0 pid = 113882 http_port = 8085 start_time = 1756692234402667 collector_name = collector_ip = 10.0.0.56:8086 collector_list= [ [ (*_iter40) = 10.0.0.56:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ComputeCpuStateTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 22 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 27 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 40 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterControlStatsTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 4 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 6 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 68 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 18 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 61, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 134, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:05:55:417.476 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011139814912, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:06:04:360.408 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011171994240, Pid 113882]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.0.0.56:8085::10.0.0.56:51994 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 255 2025-09-01 Mon 02:06:17:818.620 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011171994240, Pid 113882]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.0.0.56:8085::10.0.0.56:38570 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 255 2025-09-01 Mon 02:06:21:792.091 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011148211712, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:21:792.231 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011148211712, Pid 113882]: SANDESH: Send FAILED: 1756692381792044 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.14 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:21:792.260 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011148211712, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:21:792.284 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011148211712, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:21:792.316 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011148211712, Pid 113882]: SANDESH: Send FAILED: 1756692381792247 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.14 state = Connect event = Connect Event local_port = 38361 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:21:792.398 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011148211712, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:21:792.361 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011171994240, Pid 113882]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.14 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:06:21:792.539 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011139814912, Pid 113882]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:06:21:792.619 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011148211712, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:21:792.652 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011148211712, Pid 113882]: SANDESH: Send FAILED: 1756692381792603 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.14 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:21:792.668 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011148211712, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:21:792.686 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011148211712, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:21:792.711 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011148211712, Pid 113882]: SANDESH: Send FAILED: 1756692381792610 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.14 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:21:792.726 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011148211712, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:23:546.345 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011148211712, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:23:546.417 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011148211712, Pid 113882]: SANDESH: Send FAILED: 1756692383546292 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.14 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:23:546.433 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011148211712, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:23:546.631 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:23:546.650 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011171994240, Pid 113882]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.14 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:06:23:546.722 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: SANDESH: Send FAILED: 1756692383546575 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.14 state = Connect event = Connect Event local_port = 43689 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:23:546.805 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:23:546.963 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011148211712, Pid 113882]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:06:23:547.279 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011148211712, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:23:547.332 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011148211712, Pid 113882]: SANDESH: Send FAILED: 1756692383547114 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.14 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:23:547.354 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011148211712, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:23:547.377 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011148211712, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:23:547.405 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011148211712, Pid 113882]: SANDESH: Send FAILED: 1756692383547153 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.14 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:23:547.423 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011148211712, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:24:408.129 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011156608512, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:24:408.332 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011156608512, Pid 113882]: SANDESH: Send FAILED: 1756692384407817 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3993-1:Compute:contrail-vrouter-agent:0 sm_queue_count = 0 sm_stats= [ ev_stats= [ [ [ event = scm::EvCollectorUpdate enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvIdleHoldTimerExpired enqueues = 8 dequeues = 8 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvSandeshSend enqueues = 144 dequeues = 144 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvStart enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpConnectFail enqueues = 9 dequeues = 9 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpDeleteSession enqueues = 9 dequeues = 9 enqueue_fails = 0 dequeue_fails = 0 ], [ event = enqueues = 172 dequeues = 172 enqueue_fails = 0 dequeue_fails = 0 ], ] ] state = Idle state_since = 1756692355417197 last_state = Idle last_event = scm::EvSandeshSend last_event_at = 1756692383547374 ] max_sm_queue_count = 5 ] 2025-09-01 Mon 02:06:24:408.370 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011156608512, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:06:24:408.400 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011156608512, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:24:408.817 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011156608512, Pid 113882]: SANDESH: Send FAILED: 1756692384408022 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3993-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Idle successful_connections = 0 pid = 113882 http_port = 8085 start_time = 1756692234402667 collector_name = collector_ip = 10.0.0.56:8086 collector_list= [ [ (*_iter40) = 10.0.0.56:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ComputeCpuStateTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 23 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 28 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 44 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterControlStatsTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 4 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 6 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 76 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 20 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 67, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 144, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:06:24:408.883 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011156608512, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:06:24:414.979 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011156608512, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:24:415.428 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011156608512, Pid 113882]: SANDESH: Send FAILED: 1756692384414923 [SYS_NOTICE]: VrouterStats: data= [ name = cn-jenkins-deploy-platform-ansible-os-3993-1 in_tpkts = 82 in_bytes = 3584 out_tpkts = 82 out_bytes = 3584 exception_packets = 11 exception_packets_dropped = 11 raw_vhost_stats= [ name = vhost0 in_pkts = 82 in_bytes = 3584 out_pkts = 82 out_bytes = 3584 speed = -1 duplexity = -1 ] raw_vhost_drop_stats= [ ds_discard = 0 ds_pull = 0 ds_invalid_if = 0 ds_invalid_arp = 0 ds_trap_no_if = 0 ds_nowhere_to_go = 0 ds_flow_queue_limit_exceeded = 0 ds_flow_no_memory = 0 ds_flow_invalid_protocol = 0 ds_flow_nat_no_rflow = 0 ds_flow_action_drop = 0 ds_flow_action_invalid = 0 ds_flow_unusable = 0 ds_flow_table_full = 0 ds_interface_tx_discard = 0 ds_interface_drop = 0 ds_duplicated = 0 ds_push = 0 ds_ttl_exceeded = 0 ds_invalid_nh = 0 ds_invalid_label = 0 ds_invalid_protocol = 0 ds_interface_rx_discard = 0 ds_invalid_mcast_source = 0 ds_head_alloc_fail = 0 ds_pcow_fail = 0 ds_mcast_clone_fail = 0 ds_rewrite_fail = 0 ds_misc = 0 ds_invalid_packet = 0 ds_cksum_err = 0 ds_no_fmd = 0 ds_invalid_vnid = 0 ds_frag_err = 0 ds_invalid_source = 0 ds_mcast_df_bit = 0 ds_l2_no_route = 0 ds_vlan_fwd_tx = 0 ds_vlan_fwd_enq = 0 ds_drop_new_flow = 0 ds_flow_evict = 0 ds_trap_original = 0 ds_no_memory = 0 ds_fragment_queue_fail = 0 ds_no_frag_entry = 0 ds_icmp_error = 0 ds_clone_fail = 0 ds_drop_pkts = 0 ] raw_drop_stats= [ ds_discard = 0 ds_pull = 0 ds_invalid_if = 0 ds_invalid_arp = 0 ds_trap_no_if = 0 ds_nowhere_to_go = 0 ds_flow_queue_limit_exceeded = 0 ds_flow_no_memory = 0 ds_flow_invalid_protocol = 0 ds_flow_nat_no_rflow = 0 ds_flow_action_drop = 0 ds_flow_action_invalid = 0 ds_flow_unusable = 0 ds_flow_table_full = 0 ds_interface_tx_discard = 0 ds_interface_drop = 0 ds_duplicated = 0 ds_push = 0 ds_ttl_exceeded = 0 ds_invalid_nh = 0 ds_invalid_label = 0 ds_invalid_protocol = 0 ds_interface_rx_discard = 0 ds_invalid_mcast_source = 0 ds_head_alloc_fail = 0 ds_pcow_fail = 0 ds_mcast_clone_fail = 0 ds_rewrite_fail = 0 ds_misc = 0 ds_invalid_packet = 0 ds_cksum_err = 0 ds_no_fmd = 0 ds_invalid_vnid = 0 ds_frag_err = 0 ds_invalid_source = 0 ds_mcast_df_bit = 0 ds_l2_no_route = 0 ds_vlan_fwd_tx = 0 ds_vlan_fwd_enq = 0 ds_drop_new_flow = 0 ds_flow_evict = 0 ds_trap_original = 0 ds_no_memory = 0 ds_fragment_queue_fail = 0 ds_no_frag_entry = 0 ds_icmp_error = 0 ds_clone_fail = 0 ds_drop_pkts = 0 ] total_in_bandwidth_utilization = 9.75782e-17 flow_rate= [ added_flows = 0 max_flow_adds_per_second = 0 min_flow_adds_per_second = 0 deleted_flows = 0 max_flow_deletes_per_second = 0 min_flow_deletes_per_second = 0 active_flows = 0 hold_flows = 0 ] phy_band_in_bps= [ [ _iter260->first = ens6 _iter260->second = 18, ] ] phy_band_out_bps= [ [ _iter261->first = ens6 _iter261->second = 177, ] ] raw_phy_if_stats= [ [ _iter262->first = ens6 [ in_pkts = 5 in_bytes = 350 out_pkts = 82 out_bytes = 3584 ], ] ] raw_phy_if_drop_stats= [ [ _iter263->first = ens6 [ ds_discard = 0 ds_pull = 0 ds_invalid_if = 0 ds_invalid_arp = 0 ds_trap_no_if = 0 ds_nowhere_to_go = 0 ds_flow_queue_limit_exceeded = 0 ds_flow_no_memory = 0 ds_flow_invalid_protocol = 0 ds_flow_nat_no_rflow = 0 ds_flow_action_drop = 0 ds_flow_action_invalid = 0 ds_flow_unusable = 0 ds_flow_table_full = 0 ds_interface_tx_discard = 0 ds_interface_drop = 0 ds_duplicated = 0 ds_push = 0 ds_ttl_exceeded = 0 ds_invalid_nh = 0 ds_invalid_label = 0 ds_invalid_protocol = 0 ds_interface_rx_discard = 0 ds_invalid_mcast_source = 0 ds_head_alloc_fail = 0 ds_pcow_fail = 0 ds_mcast_clone_fail = 0 ds_rewrite_fail = 0 ds_misc = 0 ds_invalid_packet = 0 ds_cksum_err = 0 ds_no_fmd = 0 ds_invalid_vnid = 0 ds_frag_err = 0 ds_invalid_source = 0 ds_mcast_df_bit = 0 ds_l2_no_route = 0 ds_vlan_fwd_tx = 0 ds_vlan_fwd_enq = 0 ds_drop_new_flow = 0 ds_flow_evict = 0 ds_trap_original = 0 ds_no_memory = 0 ds_fragment_queue_fail = 0 ds_no_frag_entry = 0 ds_icmp_error = 0 ds_clone_fail = 0 ds_drop_pkts = 0 ], ] ] ] 2025-09-01 Mon 02:06:24:415.487 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011156608512, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: VrouterStats 2025-09-01 Mon 02:06:24:415.521 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011156608512, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:24:415.662 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011156608512, Pid 113882]: SANDESH: Send FAILED: 1756692384415007 [SYS_NOTICE]: VrouterControlStatsTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3993-1 raw_xmpp_stats= [ [ _iter306->first = 10.20.0.14 [ reconnects = 0 in_msgs = 0 out_msgs = 0 ], ] ] raw_ifmap_stats= [ [ _iter307->first = link_delete_parse_errors _iter307->second = 0, _iter307->first = link_update_parse_errors _iter307->second = 0, _iter307->first = node_delete_parse_errors _iter307->second = 0, _iter307->first = node_update_parse_errors _iter307->second = 0, ] ] raw_rt_table_size= [ [ _iter308->first = default-domain:default-project:ip-fabric:__default__ [ inet4_unicast = 4 inet4_multicast = 0 evpn = 0 bridge = 2 inet6_unicast = 2 ], _iter308->first = default-domain:default-project:ip-fabric:ip-fabric [ inet4_unicast = 2 inet4_multicast = 0 evpn = 1 bridge = 2 inet6_unicast = 0 ], ] ] ] 2025-09-01 Mon 02:06:24:415.694 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011156608512, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: VrouterControlStatsTrace 2025-09-01 Mon 02:06:25:417.308 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011135616512, Pid 113882]: Processing scm::EvIdleHoldTimerExpired in state Idle 2025-09-01 Mon 02:06:25:417.585 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011135616512, Pid 113882]: Connect : Start Connect timer 10.0.0.56:8086 2025-09-01 Mon 02:06:25:417.685 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011135616512, Pid 113882]: Processing scm::EvSandeshSend in state Connect 2025-09-01 Mon 02:06:25:417.693 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011171994240, Pid 113882]: -(-1) OnSessionEvent TCP Connect Failed 2025-09-01 Mon 02:06:25:417.710 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011135616512, Pid 113882]: SANDESH: Send FAILED: 1756692385417385 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-3993-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.14 server_addrs= [ [ (*_iter6) = 10.20.0.14:5269, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.14 server_addrs= [ [ (*_iter6) = 10.20.0.14:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.56:8086, ] ] status = Initializing description = Idle : EvIdleHoldTimerExpired -> Connect ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.14, XMPP:dns-server:10.20.0.14, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:06:25:417.801 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011171994240, Pid 113882]: -(-1):EvTcpConnectFail 2025-09-01 Mon 02:06:25:417.803 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011135616512, Pid 113882]: Wrong state: Connect for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:06:25:417.857 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011135616512, Pid 113882]: Processing scm::EvSandeshSend in state Connect 2025-09-01 Mon 02:06:25:417.978 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011135616512, Pid 113882]: SANDESH: Send FAILED: 1756692385417668 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3993-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Connect successful_connections = 0 pid = 113882 http_port = 8085 start_time = 1756692234402667 collector_name = collector_ip = 10.0.0.56:8086 collector_list= [ [ (*_iter40) = 10.0.0.56:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ComputeCpuStateTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 23 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 30 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 44 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterControlStatsTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 5 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 7 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 76 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 20 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 67, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 148, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:06:25:418.007 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011135616512, Pid 113882]: Wrong state: Connect for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:06:25:418.022 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011135616512, Pid 113882]: Processing scm::EvTcpConnectFail in state Connect 2025-09-01 Mon 02:06:25:418.030 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011135616512, Pid 113882]: Connect : EvTcpConnectFail 2025-09-01 Mon 02:06:25:418.079 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011135616512, Pid 113882]: TCP [SYS_ERR]: TcpSessionMessageLog: Session -(-1) > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:06:25:418.107 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011135616512, Pid 113882]: Idle 2025-09-01 Mon 02:06:25:418.157 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011135616512, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:25:418.173 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011135616512, Pid 113882]: SANDESH: Send FAILED: 1756692385418057 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-3993-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.14 server_addrs= [ [ (*_iter6) = 10.20.0.14:5269, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.14 server_addrs= [ [ (*_iter6) = 10.20.0.14:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.14, XMPP:dns-server:10.20.0.14, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:06:25:418.179 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011135616512, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:06:25:418.194 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011135616512, Pid 113882]: Processing scm::EvTcpDeleteSession in state Idle 2025-09-01 Mon 02:06:25:418.228 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011135616512, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:25:418.335 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011135616512, Pid 113882]: SANDESH: Send FAILED: 1756692385418147 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3993-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Idle successful_connections = 0 pid = 113882 http_port = 8085 start_time = 1756692234402667 collector_name = collector_ip = 10.0.0.56:8086 collector_list= [ [ (*_iter40) = 10.0.0.56:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ComputeCpuStateTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 24 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 31 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 45 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterControlStatsTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 5 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 7 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 76 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 20 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 68, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 150, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:06:25:418.351 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011135616512, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:06:30:669.837 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011171994240, Pid 113882]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.0.0.56:8085::10.0.0.56:36200 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 255 2025-09-01 Mon 02:06:32:477.558 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011171994240, Pid 113882]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.20.0.14:8085::10.20.0.14:48668 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 255 2025-09-01 Mon 02:06:32:498.151 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011171994240, Pid 113882]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.20.0.14:8085::10.20.0.14:48670 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 255 2025-09-01 Mon 02:06:43:462.226 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011171994240, Pid 113882]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.0.0.56:8085::10.0.0.56:37382 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 255 2025-09-01 Mon 02:06:50:682.910 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011148211712, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:50:683.000 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011148211712, Pid 113882]: SANDESH: Send FAILED: 1756692410682871 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.14 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:50:683.017 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011148211712, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:50:683.137 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011156608512, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:50:683.156 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011171994240, Pid 113882]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.14 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:06:50:683.286 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011156608512, Pid 113882]: SANDESH: Send FAILED: 1756692410683092 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.14 state = Connect event = Connect Event local_port = 43403 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:50:683.356 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011156608512, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:50:683.449 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011156608512, Pid 113882]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:06:50:683.535 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:50:683.572 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: SANDESH: Send FAILED: 1756692410683515 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.14 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:50:683.582 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:50:683.593 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:50:683.607 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: SANDESH: Send FAILED: 1756692410683531 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.14 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:50:683.628 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:52:437.503 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011156608512, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:52:437.680 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011156608512, Pid 113882]: SANDESH: Send FAILED: 1756692412437438 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.14 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:52:437.713 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011156608512, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:52:437.875 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011171994240, Pid 113882]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.14 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:06:52:437.900 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011139814912, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:52:438.003 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011139814912, Pid 113882]: SANDESH: Send FAILED: 1756692412437824 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.14 state = Connect event = Connect Event local_port = 46039 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:52:438.029 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011139814912, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:52:438.028 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011160806912, Pid 113882]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:06:52:438.335 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011135616512, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:52:438.376 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011135616512, Pid 113882]: SANDESH: Send FAILED: 1756692412438215 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.14 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:52:438.396 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011135616512, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:52:438.427 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011135616512, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:52:438.454 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011135616512, Pid 113882]: SANDESH: Send FAILED: 1756692412438280 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.14 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:52:438.472 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011135616512, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:54:408.391 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:54:408.521 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: SANDESH: Send FAILED: 1756692414408217 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3993-1:Compute:contrail-vrouter-agent:0 sm_queue_count = 0 sm_stats= [ ev_stats= [ [ [ event = scm::EvCollectorUpdate enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvIdleHoldTimerExpired enqueues = 9 dequeues = 9 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvSandeshSend enqueues = 160 dequeues = 160 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvStart enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpConnectFail enqueues = 10 dequeues = 10 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpDeleteSession enqueues = 10 dequeues = 10 enqueue_fails = 0 dequeue_fails = 0 ], [ event = enqueues = 191 dequeues = 191 enqueue_fails = 0 dequeue_fails = 0 ], ] ] state = Idle state_since = 1756692385418106 last_state = Idle last_event = scm::EvSandeshSend last_event_at = 1756692412438423 ] max_sm_queue_count = 5 ] 2025-09-01 Mon 02:06:54:408.553 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:06:54:408.565 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:54:408.772 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: SANDESH: Send FAILED: 1756692414408342 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3993-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Idle successful_connections = 0 pid = 113882 http_port = 8085 start_time = 1756692234402667 collector_name = collector_ip = 10.0.0.56:8086 collector_list= [ [ (*_iter40) = 10.0.0.56:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ComputeCpuStateTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 25 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 32 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 51 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterControlStatsTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 5 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 7 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 84 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 22 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 76, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 160, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:06:54:408.791 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:06:54:415.555 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011135616512, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:54:415.776 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011135616512, Pid 113882]: SANDESH: Send FAILED: 1756692414415394 [SYS_NOTICE]: VrouterStats: data= [ name = cn-jenkins-deploy-platform-ansible-os-3993-1 cpu_info= [ num_cpu = 8 sys_mem_info= [ total = 32 used = 0 free = 0 buffers = 0 cached = 3882065592 node_type = ] meminfo= [ virt = 32598 peakvirt = 0 res = 0 ] cpuload= [ one_min_avg = 0.2125 five_min_avg = 0.275 fifteen_min_avg = 0.25375 ] cpu_share = 0.48125 ] ] 2025-09-01 Mon 02:06:54:415.815 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011135616512, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: VrouterStats 2025-09-01 Mon 02:06:54:415.844 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011135616512, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:54:415.891 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011135616512, Pid 113882]: SANDESH: Send FAILED: 1756692414415466 [SYS_NOTICE]: ComputeCpuStateTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3993-1 cpu_info= [ [ [ mem_virt = 32598 cpu_share = 0.48125 used_sys_mem = 412901704 one_min_cpuload = 0.2125 mem_res = 0 ], ] ] ] 2025-09-01 Mon 02:06:54:415.913 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011135616512, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: ComputeCpuStateTrace 2025-09-01 Mon 02:06:54:415.935 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011135616512, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:54:416.492 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011135616512, Pid 113882]: SANDESH: Send FAILED: 1756692414415517 [SYS_NOTICE]: VrouterStats: data= [ name = cn-jenkins-deploy-platform-ansible-os-3993-1 in_tpkts = 97 in_bytes = 4214 out_tpkts = 98 out_bytes = 4284 raw_vhost_stats= [ name = vhost0 in_pkts = 97 in_bytes = 4214 out_pkts = 98 out_bytes = 4284 speed = -1 duplexity = -1 ] raw_vhost_drop_stats= [ ds_discard = 0 ds_pull = 0 ds_invalid_if = 0 ds_invalid_arp = 0 ds_trap_no_if = 0 ds_nowhere_to_go = 0 ds_flow_queue_limit_exceeded = 0 ds_flow_no_memory = 0 ds_flow_invalid_protocol = 0 ds_flow_nat_no_rflow = 0 ds_flow_action_drop = 0 ds_flow_action_invalid = 0 ds_flow_unusable = 0 ds_flow_table_full = 0 ds_interface_tx_discard = 0 ds_interface_drop = 0 ds_duplicated = 0 ds_push = 0 ds_ttl_exceeded = 0 ds_invalid_nh = 0 ds_invalid_label = 0 ds_invalid_protocol = 0 ds_interface_rx_discard = 0 ds_invalid_mcast_source = 0 ds_head_alloc_fail = 0 ds_pcow_fail = 0 ds_mcast_clone_fail = 0 ds_rewrite_fail = 0 ds_misc = 0 ds_invalid_packet = 0 ds_cksum_err = 0 ds_no_fmd = 0 ds_invalid_vnid = 0 ds_frag_err = 0 ds_invalid_source = 0 ds_mcast_df_bit = 0 ds_l2_no_route = 0 ds_vlan_fwd_tx = 0 ds_vlan_fwd_enq = 0 ds_drop_new_flow = 0 ds_flow_evict = 0 ds_trap_original = 0 ds_no_memory = 0 ds_fragment_queue_fail = 0 ds_no_frag_entry = 0 ds_icmp_error = 0 ds_clone_fail = 0 ds_drop_pkts = 0 ] raw_drop_stats= [ ds_discard = 0 ds_pull = 0 ds_invalid_if = 0 ds_invalid_arp = 0 ds_trap_no_if = 0 ds_nowhere_to_go = 0 ds_flow_queue_limit_exceeded = 0 ds_flow_no_memory = 0 ds_flow_invalid_protocol = 0 ds_flow_nat_no_rflow = 0 ds_flow_action_drop = 0 ds_flow_action_invalid = 0 ds_flow_unusable = 0 ds_flow_table_full = 0 ds_interface_tx_discard = 0 ds_interface_drop = 0 ds_duplicated = 0 ds_push = 0 ds_ttl_exceeded = 0 ds_invalid_nh = 0 ds_invalid_label = 0 ds_invalid_protocol = 0 ds_interface_rx_discard = 0 ds_invalid_mcast_source = 0 ds_head_alloc_fail = 0 ds_pcow_fail = 0 ds_mcast_clone_fail = 0 ds_rewrite_fail = 0 ds_misc = 0 ds_invalid_packet = 0 ds_cksum_err = 0 ds_no_fmd = 0 ds_invalid_vnid = 0 ds_frag_err = 0 ds_invalid_source = 0 ds_mcast_df_bit = 0 ds_l2_no_route = 0 ds_vlan_fwd_tx = 0 ds_vlan_fwd_enq = 0 ds_drop_new_flow = 0 ds_flow_evict = 0 ds_trap_original = 0 ds_no_memory = 0 ds_fragment_queue_fail = 0 ds_no_frag_entry = 0 ds_icmp_error = 0 ds_clone_fail = 0 ds_drop_pkts = 0 ] flow_rate= [ added_flows = 0 max_flow_adds_per_second = 0 min_flow_adds_per_second = 0 deleted_flows = 0 max_flow_deletes_per_second = 0 min_flow_deletes_per_second = 0 active_flows = 0 hold_flows = 0 ] raw_phy_if_stats= [ [ _iter262->first = ens6 [ in_pkts = 6 in_bytes = 420 out_pkts = 97 out_bytes = 4214 ], ] ] raw_phy_if_drop_stats= [ [ _iter263->first = ens6 [ ds_discard = 0 ds_pull = 0 ds_invalid_if = 0 ds_invalid_arp = 0 ds_trap_no_if = 0 ds_nowhere_to_go = 0 ds_flow_queue_limit_exceeded = 0 ds_flow_no_memory = 0 ds_flow_invalid_protocol = 0 ds_flow_nat_no_rflow = 0 ds_flow_action_drop = 0 ds_flow_action_invalid = 0 ds_flow_unusable = 0 ds_flow_table_full = 0 ds_interface_tx_discard = 0 ds_interface_drop = 0 ds_duplicated = 0 ds_push = 0 ds_ttl_exceeded = 0 ds_invalid_nh = 0 ds_invalid_label = 0 ds_invalid_protocol = 0 ds_interface_rx_discard = 0 ds_invalid_mcast_source = 0 ds_head_alloc_fail = 0 ds_pcow_fail = 0 ds_mcast_clone_fail = 0 ds_rewrite_fail = 0 ds_misc = 0 ds_invalid_packet = 0 ds_cksum_err = 0 ds_no_fmd = 0 ds_invalid_vnid = 0 ds_frag_err = 0 ds_invalid_source = 0 ds_mcast_df_bit = 0 ds_l2_no_route = 0 ds_vlan_fwd_tx = 0 ds_vlan_fwd_enq = 0 ds_drop_new_flow = 0 ds_flow_evict = 0 ds_trap_original = 0 ds_no_memory = 0 ds_fragment_queue_fail = 0 ds_no_frag_entry = 0 ds_icmp_error = 0 ds_clone_fail = 0 ds_drop_pkts = 0 ], ] ] ] 2025-09-01 Mon 02:06:54:416.539 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011135616512, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: VrouterStats 2025-09-01 Mon 02:06:54:416.578 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011135616512, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:54:416.653 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011135616512, Pid 113882]: SANDESH: Send FAILED: 1756692414415577 [SYS_NOTICE]: VrouterControlStatsTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3993-1 raw_xmpp_stats= [ [ _iter306->first = 10.20.0.14 [ reconnects = 0 in_msgs = 0 out_msgs = 0 ], ] ] raw_ifmap_stats= [ [ _iter307->first = link_delete_parse_errors _iter307->second = 0, _iter307->first = link_update_parse_errors _iter307->second = 0, _iter307->first = node_delete_parse_errors _iter307->second = 0, _iter307->first = node_update_parse_errors _iter307->second = 0, ] ] raw_rt_table_size= [ [ _iter308->first = default-domain:default-project:ip-fabric:__default__ [ inet4_unicast = 4 inet4_multicast = 0 evpn = 0 bridge = 2 inet6_unicast = 2 ], _iter308->first = default-domain:default-project:ip-fabric:ip-fabric [ inet4_unicast = 2 inet4_multicast = 0 evpn = 1 bridge = 2 inet6_unicast = 0 ], ] ] ] 2025-09-01 Mon 02:06:54:416.673 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011135616512, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: VrouterControlStatsTrace 2025-09-01 Mon 02:06:55:418.370 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011156608512, Pid 113882]: Processing scm::EvIdleHoldTimerExpired in state Idle 2025-09-01 Mon 02:06:55:419.013 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011156608512, Pid 113882]: Connect : Start Connect timer 10.0.0.56:8086 2025-09-01 Mon 02:06:55:419.093 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011171994240, Pid 113882]: -(-1) OnSessionEvent TCP Connect Failed 2025-09-01 Mon 02:06:55:419.213 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011171994240, Pid 113882]: -(-1):EvTcpConnectFail 2025-09-01 Mon 02:06:55:419.293 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011156608512, Pid 113882]: Processing scm::EvSandeshSend in state Connect 2025-09-01 Mon 02:06:55:419.373 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011156608512, Pid 113882]: SANDESH: Send FAILED: 1756692415418608 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-3993-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.14 server_addrs= [ [ (*_iter6) = 10.20.0.14:5269, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.14 server_addrs= [ [ (*_iter6) = 10.20.0.14:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.56:8086, ] ] status = Initializing description = Idle : EvIdleHoldTimerExpired -> Connect ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.14, XMPP:dns-server:10.20.0.14, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:06:55:419.400 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011156608512, Pid 113882]: Wrong state: Connect for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:06:55:419.429 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011156608512, Pid 113882]: Processing scm::EvTcpConnectFail in state Connect 2025-09-01 Mon 02:06:55:419.453 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011156608512, Pid 113882]: Connect : EvTcpConnectFail 2025-09-01 Mon 02:06:55:419.606 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011156608512, Pid 113882]: TCP [SYS_ERR]: TcpSessionMessageLog: Session -(-1) > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:06:55:419.706 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011156608512, Pid 113882]: Idle 2025-09-01 Mon 02:06:55:419.897 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011156608512, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:55:420.368 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011156608512, Pid 113882]: SANDESH: Send FAILED: 1756692415419237 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3993-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Connect successful_connections = 0 pid = 113882 http_port = 8085 start_time = 1756692234402667 collector_name = collector_ip = 10.0.0.56:8086 collector_list= [ [ (*_iter40) = 10.0.0.56:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ComputeCpuStateTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 3 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 25 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 34 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 51 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterControlStatsTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 6 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 9 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 84 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 22 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 76, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 166, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:06:55:420.493 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011156608512, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:06:55:420.535 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011156608512, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:55:420.615 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011156608512, Pid 113882]: SANDESH: Send FAILED: 1756692415419529 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-3993-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.14 server_addrs= [ [ (*_iter6) = 10.20.0.14:5269, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.14 server_addrs= [ [ (*_iter6) = 10.20.0.14:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.14, XMPP:dns-server:10.20.0.14, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:06:55:420.645 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011156608512, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:06:55:420.681 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011156608512, Pid 113882]: Processing scm::EvTcpDeleteSession in state Idle 2025-09-01 Mon 02:06:55:420.754 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011156608512, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:55:421.144 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011156608512, Pid 113882]: SANDESH: Send FAILED: 1756692415419856 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3993-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Idle successful_connections = 0 pid = 113882 http_port = 8085 start_time = 1756692234402667 collector_name = collector_ip = 10.0.0.56:8086 collector_list= [ [ (*_iter40) = 10.0.0.56:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ComputeCpuStateTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 3 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 26 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 34 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 52 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterControlStatsTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 6 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 9 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 84 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 22 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 77, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 167, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:06:55:421.237 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011156608512, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:06:56:359.084 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011171994240, Pid 113882]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.0.0.56:8085::10.0.0.56:52406 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 255 2025-09-01 Mon 02:07:09:192.026 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011171994240, Pid 113882]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.0.0.56:8085::10.0.0.56:37018 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 255 2025-09-01 Mon 02:07:18:493.981 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011160806912, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:18:494.138 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011160806912, Pid 113882]: SANDESH: Send FAILED: 1756692438493913 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.14 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:18:494.209 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011160806912, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:18:494.279 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011160806912, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:18:494.340 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011160806912, Pid 113882]: SANDESH: Send FAILED: 1756692438494257 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.14 state = Connect event = Connect Event local_port = 42267 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:18:494.365 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011171994240, Pid 113882]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.14 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:07:18:494.369 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011160806912, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:18:494.659 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011160806912, Pid 113882]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:07:18:494.768 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011148211712, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:18:494.860 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011148211712, Pid 113882]: SANDESH: Send FAILED: 1756692438494734 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.14 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:18:494.889 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011148211712, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:18:494.914 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011148211712, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:18:494.945 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011148211712, Pid 113882]: SANDESH: Send FAILED: 1756692438494751 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.14 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:18:494.963 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011148211712, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:19:438.596 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011135616512, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:19:438.670 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011135616512, Pid 113882]: SANDESH: Send FAILED: 1756692439438546 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.14 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:19:438.704 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011135616512, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:19:438.852 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011139814912, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:19:438.869 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011171994240, Pid 113882]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.14 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:07:19:438.910 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011139814912, Pid 113882]: SANDESH: Send FAILED: 1756692439438819 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.14 state = Connect event = Connect Event local_port = 44639 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:19:438.957 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011139814912, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:19:438.988 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011148211712, Pid 113882]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:07:19:439.106 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:19:439.234 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: SANDESH: Send FAILED: 1756692439439038 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.14 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:19:439.275 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:19:439.314 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:19:439.359 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: SANDESH: Send FAILED: 1756692439439061 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.14 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:19:439.380 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:22:551.447 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011171994240, Pid 113882]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.0.0.56:8085::10.0.0.56:33278 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 255 2025-09-01 Mon 02:07:24:408.965 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:24:409.137 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: SANDESH: Send FAILED: 1756692444408619 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3993-1:Compute:contrail-vrouter-agent:0 sm_queue_count = 0 sm_stats= [ ev_stats= [ [ [ event = scm::EvCollectorUpdate enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvIdleHoldTimerExpired enqueues = 10 dequeues = 10 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvSandeshSend enqueues = 178 dequeues = 178 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvStart enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpConnectFail enqueues = 11 dequeues = 11 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpDeleteSession enqueues = 11 dequeues = 11 enqueue_fails = 0 dequeue_fails = 0 ], [ event = enqueues = 212 dequeues = 212 enqueue_fails = 0 dequeue_fails = 0 ], ] ] state = Idle state_since = 1756692415419703 last_state = Idle last_event = scm::EvSandeshSend last_event_at = 1756692439439307 ] max_sm_queue_count = 5 ] 2025-09-01 Mon 02:07:24:409.213 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:07:24:409.240 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:24:409.628 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: SANDESH: Send FAILED: 1756692444408825 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3993-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Idle successful_connections = 0 pid = 113882 http_port = 8085 start_time = 1756692234402667 collector_name = collector_ip = 10.0.0.56:8086 collector_list= [ [ (*_iter40) = 10.0.0.56:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ComputeCpuStateTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 3 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 27 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 36 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 57 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterControlStatsTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 6 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 9 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 92 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 24 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 84, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 178, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:07:24:409.677 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:07:24:416.089 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011160806912, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:24:416.578 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011160806912, Pid 113882]: SANDESH: Send FAILED: 1756692444416019 [SYS_NOTICE]: VrouterStats: data= [ name = cn-jenkins-deploy-platform-ansible-os-3993-1 in_tpkts = 112 in_bytes = 4844 out_tpkts = 113 out_bytes = 4914 raw_vhost_stats= [ name = vhost0 in_pkts = 112 in_bytes = 4844 out_pkts = 113 out_bytes = 4914 speed = -1 duplexity = -1 ] raw_vhost_drop_stats= [ ds_discard = 0 ds_pull = 0 ds_invalid_if = 0 ds_invalid_arp = 0 ds_trap_no_if = 0 ds_nowhere_to_go = 0 ds_flow_queue_limit_exceeded = 0 ds_flow_no_memory = 0 ds_flow_invalid_protocol = 0 ds_flow_nat_no_rflow = 0 ds_flow_action_drop = 0 ds_flow_action_invalid = 0 ds_flow_unusable = 0 ds_flow_table_full = 0 ds_interface_tx_discard = 0 ds_interface_drop = 0 ds_duplicated = 0 ds_push = 0 ds_ttl_exceeded = 0 ds_invalid_nh = 0 ds_invalid_label = 0 ds_invalid_protocol = 0 ds_interface_rx_discard = 0 ds_invalid_mcast_source = 0 ds_head_alloc_fail = 0 ds_pcow_fail = 0 ds_mcast_clone_fail = 0 ds_rewrite_fail = 0 ds_misc = 0 ds_invalid_packet = 0 ds_cksum_err = 0 ds_no_fmd = 0 ds_invalid_vnid = 0 ds_frag_err = 0 ds_invalid_source = 0 ds_mcast_df_bit = 0 ds_l2_no_route = 0 ds_vlan_fwd_tx = 0 ds_vlan_fwd_enq = 0 ds_drop_new_flow = 0 ds_flow_evict = 0 ds_trap_original = 0 ds_no_memory = 0 ds_fragment_queue_fail = 0 ds_no_frag_entry = 0 ds_icmp_error = 0 ds_clone_fail = 0 ds_drop_pkts = 0 ] raw_drop_stats= [ ds_discard = 0 ds_pull = 0 ds_invalid_if = 0 ds_invalid_arp = 0 ds_trap_no_if = 0 ds_nowhere_to_go = 0 ds_flow_queue_limit_exceeded = 0 ds_flow_no_memory = 0 ds_flow_invalid_protocol = 0 ds_flow_nat_no_rflow = 0 ds_flow_action_drop = 0 ds_flow_action_invalid = 0 ds_flow_unusable = 0 ds_flow_table_full = 0 ds_interface_tx_discard = 0 ds_interface_drop = 0 ds_duplicated = 0 ds_push = 0 ds_ttl_exceeded = 0 ds_invalid_nh = 0 ds_invalid_label = 0 ds_invalid_protocol = 0 ds_interface_rx_discard = 0 ds_invalid_mcast_source = 0 ds_head_alloc_fail = 0 ds_pcow_fail = 0 ds_mcast_clone_fail = 0 ds_rewrite_fail = 0 ds_misc = 0 ds_invalid_packet = 0 ds_cksum_err = 0 ds_no_fmd = 0 ds_invalid_vnid = 0 ds_frag_err = 0 ds_invalid_source = 0 ds_mcast_df_bit = 0 ds_l2_no_route = 0 ds_vlan_fwd_tx = 0 ds_vlan_fwd_enq = 0 ds_drop_new_flow = 0 ds_flow_evict = 0 ds_trap_original = 0 ds_no_memory = 0 ds_fragment_queue_fail = 0 ds_no_frag_entry = 0 ds_icmp_error = 0 ds_clone_fail = 0 ds_drop_pkts = 0 ] total_in_bandwidth_utilization = 4.87891e-17 total_out_bandwidth_utilization = 9.1073e-16 flow_rate= [ added_flows = 0 max_flow_adds_per_second = 0 min_flow_adds_per_second = 0 deleted_flows = 0 max_flow_deletes_per_second = 0 min_flow_deletes_per_second = 0 active_flows = 0 hold_flows = 0 ] phy_band_in_bps= [ [ _iter260->first = ens6 _iter260->second = 9, ] ] phy_band_out_bps= [ [ _iter261->first = ens6 _iter261->second = 168, ] ] raw_phy_if_stats= [ [ _iter262->first = ens6 [ in_pkts = 6 in_bytes = 420 out_pkts = 112 out_bytes = 4844 ], ] ] raw_phy_if_drop_stats= [ [ _iter263->first = ens6 [ ds_discard = 0 ds_pull = 0 ds_invalid_if = 0 ds_invalid_arp = 0 ds_trap_no_if = 0 ds_nowhere_to_go = 0 ds_flow_queue_limit_exceeded = 0 ds_flow_no_memory = 0 ds_flow_invalid_protocol = 0 ds_flow_nat_no_rflow = 0 ds_flow_action_drop = 0 ds_flow_action_invalid = 0 ds_flow_unusable = 0 ds_flow_table_full = 0 ds_interface_tx_discard = 0 ds_interface_drop = 0 ds_duplicated = 0 ds_push = 0 ds_ttl_exceeded = 0 ds_invalid_nh = 0 ds_invalid_label = 0 ds_invalid_protocol = 0 ds_interface_rx_discard = 0 ds_invalid_mcast_source = 0 ds_head_alloc_fail = 0 ds_pcow_fail = 0 ds_mcast_clone_fail = 0 ds_rewrite_fail = 0 ds_misc = 0 ds_invalid_packet = 0 ds_cksum_err = 0 ds_no_fmd = 0 ds_invalid_vnid = 0 ds_frag_err = 0 ds_invalid_source = 0 ds_mcast_df_bit = 0 ds_l2_no_route = 0 ds_vlan_fwd_tx = 0 ds_vlan_fwd_enq = 0 ds_drop_new_flow = 0 ds_flow_evict = 0 ds_trap_original = 0 ds_no_memory = 0 ds_fragment_queue_fail = 0 ds_no_frag_entry = 0 ds_icmp_error = 0 ds_clone_fail = 0 ds_drop_pkts = 0 ], ] ] ] 2025-09-01 Mon 02:07:24:416.664 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011160806912, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: VrouterStats 2025-09-01 Mon 02:07:24:416.706 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011160806912, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:24:416.799 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011160806912, Pid 113882]: SANDESH: Send FAILED: 1756692444416111 [SYS_NOTICE]: VrouterControlStatsTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3993-1 raw_xmpp_stats= [ [ _iter306->first = 10.20.0.14 [ reconnects = 0 in_msgs = 0 out_msgs = 0 ], ] ] raw_ifmap_stats= [ [ _iter307->first = link_delete_parse_errors _iter307->second = 0, _iter307->first = link_update_parse_errors _iter307->second = 0, _iter307->first = node_delete_parse_errors _iter307->second = 0, _iter307->first = node_update_parse_errors _iter307->second = 0, ] ] raw_rt_table_size= [ [ _iter308->first = default-domain:default-project:ip-fabric:__default__ [ inet4_unicast = 4 inet4_multicast = 0 evpn = 0 bridge = 2 inet6_unicast = 2 ], _iter308->first = default-domain:default-project:ip-fabric:ip-fabric [ inet4_unicast = 2 inet4_multicast = 0 evpn = 1 bridge = 2 inet6_unicast = 0 ], ] ] ] 2025-09-01 Mon 02:07:24:416.822 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011160806912, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: VrouterControlStatsTrace 2025-09-01 Mon 02:07:25:419.882 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011156608512, Pid 113882]: Processing scm::EvIdleHoldTimerExpired in state Idle 2025-09-01 Mon 02:07:25:420.233 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011156608512, Pid 113882]: Connect : Start Connect timer 10.0.0.56:8086 2025-09-01 Mon 02:07:25:420.301 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011171994240, Pid 113882]: -(-1) OnSessionEvent TCP Connect Failed 2025-09-01 Mon 02:07:25:420.369 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011156608512, Pid 113882]: Processing scm::EvSandeshSend in state Connect 2025-09-01 Mon 02:07:25:420.373 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011171994240, Pid 113882]: -(-1):EvTcpConnectFail 2025-09-01 Mon 02:07:25:420.457 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011156608512, Pid 113882]: SANDESH: Send FAILED: 1756692445419984 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-3993-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.14 server_addrs= [ [ (*_iter6) = 10.20.0.14:5269, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.14 server_addrs= [ [ (*_iter6) = 10.20.0.14:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.56:8086, ] ] status = Initializing description = Idle : EvIdleHoldTimerExpired -> Connect ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.14, XMPP:dns-server:10.20.0.14, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:07:25:420.610 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011156608512, Pid 113882]: Wrong state: Connect for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:07:25:420.626 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011156608512, Pid 113882]: Processing scm::EvSandeshSend in state Connect 2025-09-01 Mon 02:07:25:420.829 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011156608512, Pid 113882]: SANDESH: Send FAILED: 1756692445420342 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3993-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Connect successful_connections = 0 pid = 113882 http_port = 8085 start_time = 1756692234402667 collector_name = collector_ip = 10.0.0.56:8086 collector_list= [ [ (*_iter40) = 10.0.0.56:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ComputeCpuStateTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 3 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 27 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 38 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 57 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterControlStatsTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 7 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 10 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 92 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 24 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 84, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 182, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:07:25:420.858 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011156608512, Pid 113882]: Wrong state: Connect for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:07:25:420.873 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011156608512, Pid 113882]: Processing scm::EvTcpConnectFail in state Connect 2025-09-01 Mon 02:07:25:420.883 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011156608512, Pid 113882]: Connect : EvTcpConnectFail 2025-09-01 Mon 02:07:25:420.944 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011156608512, Pid 113882]: TCP [SYS_ERR]: TcpSessionMessageLog: Session -(-1) > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:07:25:420.982 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011156608512, Pid 113882]: Idle 2025-09-01 Mon 02:07:25:421.062 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011156608512, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:25:421.090 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011156608512, Pid 113882]: SANDESH: Send FAILED: 1756692445420914 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-3993-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.14 server_addrs= [ [ (*_iter6) = 10.20.0.14:5269, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.14 server_addrs= [ [ (*_iter6) = 10.20.0.14:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.14, XMPP:dns-server:10.20.0.14, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:07:25:421.107 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011156608512, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:07:25:421.119 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011156608512, Pid 113882]: Processing scm::EvTcpDeleteSession in state Idle 2025-09-01 Mon 02:07:25:421.156 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011156608512, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:25:421.361 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011156608512, Pid 113882]: SANDESH: Send FAILED: 1756692445421045 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3993-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Idle successful_connections = 0 pid = 113882 http_port = 8085 start_time = 1756692234402667 collector_name = collector_ip = 10.0.0.56:8086 collector_list= [ [ (*_iter40) = 10.0.0.56:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ComputeCpuStateTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 3 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 28 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 39 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 58 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterControlStatsTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 7 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 10 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 92 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 24 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 85, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 184, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:07:25:421.668 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011156608512, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:07:33:377.358 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011171994240, Pid 113882]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.20.0.14:8085::10.20.0.14:39436 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 255 2025-09-01 Mon 02:07:33:389.648 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011171994240, Pid 113882]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.20.0.14:8085::10.20.0.14:39450 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 255 2025-09-01 Mon 02:07:35:555.080 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011171994240, Pid 113882]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.0.0.56:8085::10.0.0.56:44946 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 255 2025-09-01 Mon 02:07:45:765.149 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011152410112, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:45:765.347 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011152410112, Pid 113882]: SANDESH: Send FAILED: 1756692465765079 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.14 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:45:765.388 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011152410112, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:45:765.567 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011171994240, Pid 113882]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.14 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:07:45:765.614 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011148211712, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:45:765.688 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011135616512, Pid 113882]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:07:45:765.738 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011148211712, Pid 113882]: SANDESH: Send FAILED: 1756692465765528 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.14 state = Connect event = Connect Event local_port = 36587 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:45:765.803 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011148211712, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:45:765.838 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011148211712, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:45:765.877 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011148211712, Pid 113882]: SANDESH: Send FAILED: 1756692465765738 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.14 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:45:765.902 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011148211712, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:45:765.929 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011148211712, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:45:765.954 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011148211712, Pid 113882]: SANDESH: Send FAILED: 1756692465765748 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.14 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:45:765.968 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011148211712, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:48:538.984 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011171994240, Pid 113882]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.0.0.56:8085::10.0.0.56:42792 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 255 2025-09-01 Mon 02:07:48:599.409 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011160806912, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:48:599.510 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011160806912, Pid 113882]: SANDESH: Send FAILED: 1756692468599343 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.14 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:48:599.534 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011160806912, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:48:599.614 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011171994240, Pid 113882]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.14 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:07:48:599.617 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011148211712, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:48:599.769 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011139814912, Pid 113882]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:07:48:599.890 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011148211712, Pid 113882]: SANDESH: Send FAILED: 1756692468599579 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.14 state = Connect event = Connect Event local_port = 35533 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:48:599.935 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011148211712, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:48:600.019 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011148211712, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:48:600.076 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011148211712, Pid 113882]: SANDESH: Send FAILED: 1756692468599912 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.14 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:48:600.106 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011148211712, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:48:600.140 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011148211712, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:48:600.217 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011148211712, Pid 113882]: SANDESH: Send FAILED: 1756692468599930 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.14 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:48:600.250 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011148211712, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:54:409.315 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011160806912, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:54:409.411 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011160806912, Pid 113882]: SANDESH: Send FAILED: 1756692474409108 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3993-1:Compute:contrail-vrouter-agent:0 sm_queue_count = 0 sm_stats= [ ev_stats= [ [ [ event = scm::EvCollectorUpdate enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvIdleHoldTimerExpired enqueues = 11 dequeues = 11 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvSandeshSend enqueues = 194 dequeues = 194 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvStart enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpConnectFail enqueues = 12 dequeues = 12 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpDeleteSession enqueues = 12 dequeues = 12 enqueue_fails = 0 dequeue_fails = 0 ], [ event = enqueues = 231 dequeues = 231 enqueue_fails = 0 dequeue_fails = 0 ], ] ] state = Idle state_since = 1756692445420981 last_state = Idle last_event = scm::EvSandeshSend last_event_at = 1756692468600133 ] max_sm_queue_count = 5 ] 2025-09-01 Mon 02:07:54:409.427 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011160806912, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:07:54:409.437 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011160806912, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:54:409.605 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011160806912, Pid 113882]: SANDESH: Send FAILED: 1756692474409278 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3993-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Idle successful_connections = 0 pid = 113882 http_port = 8085 start_time = 1756692234402667 collector_name = collector_ip = 10.0.0.56:8086 collector_list= [ [ (*_iter40) = 10.0.0.56:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ComputeCpuStateTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 3 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 29 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 40 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 64 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterControlStatsTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 7 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 10 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 100 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 26 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 93, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 194, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:07:54:409.630 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011160806912, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:07:54:416.415 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011139814912, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:54:416.689 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011139814912, Pid 113882]: SANDESH: Send FAILED: 1756692474416358 [SYS_NOTICE]: VrouterStats: data= [ name = cn-jenkins-deploy-platform-ansible-os-3993-1 cpu_info= [ num_cpu = 8 sys_mem_info= [ total = 32 used = 0 free = 0 buffers = 0 cached = 3882065592 node_type = ] meminfo= [ virt = 32598 peakvirt = 0 res = 0 ] cpuload= [ one_min_avg = 0.2275 five_min_avg = 0.27125 fifteen_min_avg = 0.25375 ] cpu_share = 0.470833 ] ] 2025-09-01 Mon 02:07:54:416.721 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011139814912, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: VrouterStats 2025-09-01 Mon 02:07:54:416.750 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011139814912, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:54:416.798 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011139814912, Pid 113882]: SANDESH: Send FAILED: 1756692474416377 [SYS_NOTICE]: ComputeCpuStateTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3993-1 cpu_info= [ [ [ mem_virt = 32598 cpu_share = 0.470833 used_sys_mem = 412901704 one_min_cpuload = 0.2275 mem_res = 0 ], ] ] ] 2025-09-01 Mon 02:07:54:416.818 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011139814912, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: ComputeCpuStateTrace 2025-09-01 Mon 02:07:54:416.845 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011139814912, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:54:417.276 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011139814912, Pid 113882]: SANDESH: Send FAILED: 1756692474416405 [SYS_NOTICE]: VrouterStats: data= [ name = cn-jenkins-deploy-platform-ansible-os-3993-1 in_tpkts = 127 in_bytes = 5474 out_tpkts = 129 out_bytes = 5614 raw_vhost_stats= [ name = vhost0 in_pkts = 127 in_bytes = 5474 out_pkts = 129 out_bytes = 5614 speed = -1 duplexity = -1 ] raw_vhost_drop_stats= [ ds_discard = 0 ds_pull = 0 ds_invalid_if = 0 ds_invalid_arp = 0 ds_trap_no_if = 0 ds_nowhere_to_go = 0 ds_flow_queue_limit_exceeded = 0 ds_flow_no_memory = 0 ds_flow_invalid_protocol = 0 ds_flow_nat_no_rflow = 0 ds_flow_action_drop = 0 ds_flow_action_invalid = 0 ds_flow_unusable = 0 ds_flow_table_full = 0 ds_interface_tx_discard = 0 ds_interface_drop = 0 ds_duplicated = 0 ds_push = 0 ds_ttl_exceeded = 0 ds_invalid_nh = 0 ds_invalid_label = 0 ds_invalid_protocol = 0 ds_interface_rx_discard = 0 ds_invalid_mcast_source = 0 ds_head_alloc_fail = 0 ds_pcow_fail = 0 ds_mcast_clone_fail = 0 ds_rewrite_fail = 0 ds_misc = 0 ds_invalid_packet = 0 ds_cksum_err = 0 ds_no_fmd = 0 ds_invalid_vnid = 0 ds_frag_err = 0 ds_invalid_source = 0 ds_mcast_df_bit = 0 ds_l2_no_route = 0 ds_vlan_fwd_tx = 0 ds_vlan_fwd_enq = 0 ds_drop_new_flow = 0 ds_flow_evict = 0 ds_trap_original = 0 ds_no_memory = 0 ds_fragment_queue_fail = 0 ds_no_frag_entry = 0 ds_icmp_error = 0 ds_clone_fail = 0 ds_drop_pkts = 0 ] raw_drop_stats= [ ds_discard = 0 ds_pull = 0 ds_invalid_if = 0 ds_invalid_arp = 0 ds_trap_no_if = 0 ds_nowhere_to_go = 0 ds_flow_queue_limit_exceeded = 0 ds_flow_no_memory = 0 ds_flow_invalid_protocol = 0 ds_flow_nat_no_rflow = 0 ds_flow_action_drop = 0 ds_flow_action_invalid = 0 ds_flow_unusable = 0 ds_flow_table_full = 0 ds_interface_tx_discard = 0 ds_interface_drop = 0 ds_duplicated = 0 ds_push = 0 ds_ttl_exceeded = 0 ds_invalid_nh = 0 ds_invalid_label = 0 ds_invalid_protocol = 0 ds_interface_rx_discard = 0 ds_invalid_mcast_source = 0 ds_head_alloc_fail = 0 ds_pcow_fail = 0 ds_mcast_clone_fail = 0 ds_rewrite_fail = 0 ds_misc = 0 ds_invalid_packet = 0 ds_cksum_err = 0 ds_no_fmd = 0 ds_invalid_vnid = 0 ds_frag_err = 0 ds_invalid_source = 0 ds_mcast_df_bit = 0 ds_l2_no_route = 0 ds_vlan_fwd_tx = 0 ds_vlan_fwd_enq = 0 ds_drop_new_flow = 0 ds_flow_evict = 0 ds_trap_original = 0 ds_no_memory = 0 ds_fragment_queue_fail = 0 ds_no_frag_entry = 0 ds_icmp_error = 0 ds_clone_fail = 0 ds_drop_pkts = 0 ] flow_rate= [ added_flows = 0 max_flow_adds_per_second = 0 min_flow_adds_per_second = 0 deleted_flows = 0 max_flow_deletes_per_second = 0 min_flow_deletes_per_second = 0 active_flows = 0 hold_flows = 0 ] raw_phy_if_stats= [ [ _iter262->first = ens6 [ in_pkts = 7 in_bytes = 490 out_pkts = 127 out_bytes = 5474 ], ] ] raw_phy_if_drop_stats= [ [ _iter263->first = ens6 [ ds_discard = 0 ds_pull = 0 ds_invalid_if = 0 ds_invalid_arp = 0 ds_trap_no_if = 0 ds_nowhere_to_go = 0 ds_flow_queue_limit_exceeded = 0 ds_flow_no_memory = 0 ds_flow_invalid_protocol = 0 ds_flow_nat_no_rflow = 0 ds_flow_action_drop = 0 ds_flow_action_invalid = 0 ds_flow_unusable = 0 ds_flow_table_full = 0 ds_interface_tx_discard = 0 ds_interface_drop = 0 ds_duplicated = 0 ds_push = 0 ds_ttl_exceeded = 0 ds_invalid_nh = 0 ds_invalid_label = 0 ds_invalid_protocol = 0 ds_interface_rx_discard = 0 ds_invalid_mcast_source = 0 ds_head_alloc_fail = 0 ds_pcow_fail = 0 ds_mcast_clone_fail = 0 ds_rewrite_fail = 0 ds_misc = 0 ds_invalid_packet = 0 ds_cksum_err = 0 ds_no_fmd = 0 ds_invalid_vnid = 0 ds_frag_err = 0 ds_invalid_source = 0 ds_mcast_df_bit = 0 ds_l2_no_route = 0 ds_vlan_fwd_tx = 0 ds_vlan_fwd_enq = 0 ds_drop_new_flow = 0 ds_flow_evict = 0 ds_trap_original = 0 ds_no_memory = 0 ds_fragment_queue_fail = 0 ds_no_frag_entry = 0 ds_icmp_error = 0 ds_clone_fail = 0 ds_drop_pkts = 0 ], ] ] ] 2025-09-01 Mon 02:07:54:417.354 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011139814912, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: VrouterStats 2025-09-01 Mon 02:07:54:417.387 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011139814912, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:54:417.472 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011139814912, Pid 113882]: SANDESH: Send FAILED: 1756692474416471 [SYS_NOTICE]: VrouterControlStatsTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3993-1 raw_xmpp_stats= [ [ _iter306->first = 10.20.0.14 [ reconnects = 0 in_msgs = 0 out_msgs = 0 ], ] ] raw_ifmap_stats= [ [ _iter307->first = link_delete_parse_errors _iter307->second = 0, _iter307->first = link_update_parse_errors _iter307->second = 0, _iter307->first = node_delete_parse_errors _iter307->second = 0, _iter307->first = node_update_parse_errors _iter307->second = 0, ] ] raw_rt_table_size= [ [ _iter308->first = default-domain:default-project:ip-fabric:__default__ [ inet4_unicast = 4 inet4_multicast = 0 evpn = 0 bridge = 2 inet6_unicast = 2 ], _iter308->first = default-domain:default-project:ip-fabric:ip-fabric [ inet4_unicast = 2 inet4_multicast = 0 evpn = 1 bridge = 2 inet6_unicast = 0 ], ] ] ] 2025-09-01 Mon 02:07:54:417.495 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011139814912, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: VrouterControlStatsTrace 2025-09-01 Mon 02:07:55:421.163 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011156608512, Pid 113882]: Processing scm::EvIdleHoldTimerExpired in state Idle 2025-09-01 Mon 02:07:55:421.418 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011171994240, Pid 113882]: -(-1) OnSessionEvent TCP Connect Failed 2025-09-01 Mon 02:07:55:421.430 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011171994240, Pid 113882]: -(-1):EvTcpConnectFail 2025-09-01 Mon 02:07:55:421.445 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011156608512, Pid 113882]: Connect : Start Connect timer 10.0.0.56:8086 2025-09-01 Mon 02:07:55:421.528 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011156608512, Pid 113882]: Processing scm::EvSandeshSend in state Connect 2025-09-01 Mon 02:07:55:421.550 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011156608512, Pid 113882]: SANDESH: Send FAILED: 1756692475421241 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-3993-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.14 server_addrs= [ [ (*_iter6) = 10.20.0.14:5269, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.14 server_addrs= [ [ (*_iter6) = 10.20.0.14:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.56:8086, ] ] status = Initializing description = Idle : EvIdleHoldTimerExpired -> Connect ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.14, XMPP:dns-server:10.20.0.14, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:07:55:421.566 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011156608512, Pid 113882]: Wrong state: Connect for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:07:55:421.574 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011156608512, Pid 113882]: Processing scm::EvTcpConnectFail in state Connect 2025-09-01 Mon 02:07:55:421.580 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011156608512, Pid 113882]: Connect : EvTcpConnectFail 2025-09-01 Mon 02:07:55:421.612 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011156608512, Pid 113882]: TCP [SYS_ERR]: TcpSessionMessageLog: Session -(-1) > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:07:55:421.636 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011156608512, Pid 113882]: Idle 2025-09-01 Mon 02:07:55:421.673 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011156608512, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:55:421.788 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011156608512, Pid 113882]: SANDESH: Send FAILED: 1756692475421515 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3993-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Connect successful_connections = 0 pid = 113882 http_port = 8085 start_time = 1756692234402667 collector_name = collector_ip = 10.0.0.56:8086 collector_list= [ [ (*_iter40) = 10.0.0.56:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ComputeCpuStateTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 4 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 29 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 42 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 64 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterControlStatsTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 8 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 12 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 100 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 26 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 93, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 200, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:07:55:421.804 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011156608512, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:07:55:421.812 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011156608512, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:55:421.826 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011156608512, Pid 113882]: SANDESH: Send FAILED: 1756692475421596 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-3993-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.14 server_addrs= [ [ (*_iter6) = 10.20.0.14:5269, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.14 server_addrs= [ [ (*_iter6) = 10.20.0.14:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.14, XMPP:dns-server:10.20.0.14, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:07:55:421.831 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011156608512, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:07:55:421.837 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011156608512, Pid 113882]: Processing scm::EvTcpDeleteSession in state Idle 2025-09-01 Mon 02:07:55:421.861 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011156608512, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:55:421.959 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011156608512, Pid 113882]: SANDESH: Send FAILED: 1756692475421665 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3993-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Idle successful_connections = 0 pid = 113882 http_port = 8085 start_time = 1756692234402667 collector_name = collector_ip = 10.0.0.56:8086 collector_list= [ [ (*_iter40) = 10.0.0.56:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ComputeCpuStateTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 4 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 30 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 42 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 65 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterControlStatsTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 8 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 12 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 100 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 26 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 94, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 201, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:07:55:421.972 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011156608512, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:08:01:466.081 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011171994240, Pid 113882]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.0.0.56:8085::10.0.0.56:44656 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 255 2025-09-01 Mon 02:08:12:766.130 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011160806912, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:12:766.262 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011160806912, Pid 113882]: SANDESH: Send FAILED: 1756692492766063 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.14 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:12:766.293 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011160806912, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:12:766.485 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011156608512, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:12:766.509 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011171994240, Pid 113882]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.14 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:08:12:766.599 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011156608512, Pid 113882]: SANDESH: Send FAILED: 1756692492766419 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.14 state = Connect event = Connect Event local_port = 33959 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:12:766.713 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011156608512, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:12:766.757 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011160806912, Pid 113882]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:08:12:766.930 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:12:767.022 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: SANDESH: Send FAILED: 1756692492766830 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.14 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:12:767.048 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:12:767.099 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:12:767.129 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: SANDESH: Send FAILED: 1756692492766875 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.14 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:12:767.151 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:14:346.068 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011171994240, Pid 113882]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.0.0.56:8085::10.0.0.56:35770 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 255 2025-09-01 Mon 02:08:18:570.228 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011135616512, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:18:570.349 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011135616512, Pid 113882]: SANDESH: Send FAILED: 1756692498570153 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.14 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:18:570.393 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011135616512, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:18:570.421 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011135616512, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:18:570.451 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011171994240, Pid 113882]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.14 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:08:18:570.457 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011135616512, Pid 113882]: SANDESH: Send FAILED: 1756692498570334 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.14 state = Connect event = Connect Event local_port = 36247 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:18:570.585 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011135616512, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:18:570.669 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011135616512, Pid 113882]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:08:18:570.820 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011139814912, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:18:570.913 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011139814912, Pid 113882]: SANDESH: Send FAILED: 1756692498570733 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.14 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:18:570.941 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011139814912, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:18:570.968 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011139814912, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:18:570.997 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011139814912, Pid 113882]: SANDESH: Send FAILED: 1756692498570777 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.14 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:18:571.042 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011139814912, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:24:409.926 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011152410112, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:24:410.201 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011152410112, Pid 113882]: SANDESH: Send FAILED: 1756692504409571 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3993-1:Compute:contrail-vrouter-agent:0 sm_queue_count = 0 sm_stats= [ ev_stats= [ [ [ event = scm::EvCollectorUpdate enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvIdleHoldTimerExpired enqueues = 12 dequeues = 12 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvSandeshSend enqueues = 212 dequeues = 212 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvStart enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpConnectFail enqueues = 13 dequeues = 13 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpDeleteSession enqueues = 13 dequeues = 13 enqueue_fails = 0 dequeue_fails = 0 ], [ event = enqueues = 252 dequeues = 252 enqueue_fails = 0 dequeue_fails = 0 ], ] ] state = Idle state_since = 1756692475421635 last_state = Idle last_event = scm::EvSandeshSend last_event_at = 1756692498570964 ] max_sm_queue_count = 5 ] 2025-09-01 Mon 02:08:24:410.247 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011152410112, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:08:24:410.276 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011152410112, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:24:410.717 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011152410112, Pid 113882]: SANDESH: Send FAILED: 1756692504409772 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3993-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Idle successful_connections = 0 pid = 113882 http_port = 8085 start_time = 1756692234402667 collector_name = collector_ip = 10.0.0.56:8086 collector_list= [ [ (*_iter40) = 10.0.0.56:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ComputeCpuStateTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 4 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 31 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 44 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 69 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterControlStatsTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 8 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 12 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 108 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 28 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 100, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 212, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:08:24:410.777 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011152410112, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:08:24:416.749 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011160806912, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:24:417.297 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011160806912, Pid 113882]: SANDESH: Send FAILED: 1756692504416679 [SYS_NOTICE]: VrouterStats: data= [ name = cn-jenkins-deploy-platform-ansible-os-3993-1 in_tpkts = 143 in_bytes = 6174 out_tpkts = 144 out_bytes = 6244 raw_vhost_stats= [ name = vhost0 in_pkts = 143 in_bytes = 6174 out_pkts = 144 out_bytes = 6244 speed = -1 duplexity = -1 ] raw_vhost_drop_stats= [ ds_discard = 0 ds_pull = 0 ds_invalid_if = 0 ds_invalid_arp = 0 ds_trap_no_if = 0 ds_nowhere_to_go = 0 ds_flow_queue_limit_exceeded = 0 ds_flow_no_memory = 0 ds_flow_invalid_protocol = 0 ds_flow_nat_no_rflow = 0 ds_flow_action_drop = 0 ds_flow_action_invalid = 0 ds_flow_unusable = 0 ds_flow_table_full = 0 ds_interface_tx_discard = 0 ds_interface_drop = 0 ds_duplicated = 0 ds_push = 0 ds_ttl_exceeded = 0 ds_invalid_nh = 0 ds_invalid_label = 0 ds_invalid_protocol = 0 ds_interface_rx_discard = 0 ds_invalid_mcast_source = 0 ds_head_alloc_fail = 0 ds_pcow_fail = 0 ds_mcast_clone_fail = 0 ds_rewrite_fail = 0 ds_misc = 0 ds_invalid_packet = 0 ds_cksum_err = 0 ds_no_fmd = 0 ds_invalid_vnid = 0 ds_frag_err = 0 ds_invalid_source = 0 ds_mcast_df_bit = 0 ds_l2_no_route = 0 ds_vlan_fwd_tx = 0 ds_vlan_fwd_enq = 0 ds_drop_new_flow = 0 ds_flow_evict = 0 ds_trap_original = 0 ds_no_memory = 0 ds_fragment_queue_fail = 0 ds_no_frag_entry = 0 ds_icmp_error = 0 ds_clone_fail = 0 ds_drop_pkts = 0 ] raw_drop_stats= [ ds_discard = 0 ds_pull = 0 ds_invalid_if = 0 ds_invalid_arp = 0 ds_trap_no_if = 0 ds_nowhere_to_go = 0 ds_flow_queue_limit_exceeded = 0 ds_flow_no_memory = 0 ds_flow_invalid_protocol = 0 ds_flow_nat_no_rflow = 0 ds_flow_action_drop = 0 ds_flow_action_invalid = 0 ds_flow_unusable = 0 ds_flow_table_full = 0 ds_interface_tx_discard = 0 ds_interface_drop = 0 ds_duplicated = 0 ds_push = 0 ds_ttl_exceeded = 0 ds_invalid_nh = 0 ds_invalid_label = 0 ds_invalid_protocol = 0 ds_interface_rx_discard = 0 ds_invalid_mcast_source = 0 ds_head_alloc_fail = 0 ds_pcow_fail = 0 ds_mcast_clone_fail = 0 ds_rewrite_fail = 0 ds_misc = 0 ds_invalid_packet = 0 ds_cksum_err = 0 ds_no_fmd = 0 ds_invalid_vnid = 0 ds_frag_err = 0 ds_invalid_source = 0 ds_mcast_df_bit = 0 ds_l2_no_route = 0 ds_vlan_fwd_tx = 0 ds_vlan_fwd_enq = 0 ds_drop_new_flow = 0 ds_flow_evict = 0 ds_trap_original = 0 ds_no_memory = 0 ds_fragment_queue_fail = 0 ds_no_frag_entry = 0 ds_icmp_error = 0 ds_clone_fail = 0 ds_drop_pkts = 0 ] total_out_bandwidth_utilization = 9.59519e-16 flow_rate= [ added_flows = 0 max_flow_adds_per_second = 0 min_flow_adds_per_second = 0 deleted_flows = 0 max_flow_deletes_per_second = 0 min_flow_deletes_per_second = 0 active_flows = 0 hold_flows = 0 ] phy_band_in_bps= [ [ _iter260->first = ens6 _iter260->second = 9, ] ] phy_band_out_bps= [ [ _iter261->first = ens6 _iter261->second = 177, ] ] raw_phy_if_stats= [ [ _iter262->first = ens6 [ in_pkts = 7 in_bytes = 490 out_pkts = 143 out_bytes = 6174 ], ] ] raw_phy_if_drop_stats= [ [ _iter263->first = ens6 [ ds_discard = 0 ds_pull = 0 ds_invalid_if = 0 ds_invalid_arp = 0 ds_trap_no_if = 0 ds_nowhere_to_go = 0 ds_flow_queue_limit_exceeded = 0 ds_flow_no_memory = 0 ds_flow_invalid_protocol = 0 ds_flow_nat_no_rflow = 0 ds_flow_action_drop = 0 ds_flow_action_invalid = 0 ds_flow_unusable = 0 ds_flow_table_full = 0 ds_interface_tx_discard = 0 ds_interface_drop = 0 ds_duplicated = 0 ds_push = 0 ds_ttl_exceeded = 0 ds_invalid_nh = 0 ds_invalid_label = 0 ds_invalid_protocol = 0 ds_interface_rx_discard = 0 ds_invalid_mcast_source = 0 ds_head_alloc_fail = 0 ds_pcow_fail = 0 ds_mcast_clone_fail = 0 ds_rewrite_fail = 0 ds_misc = 0 ds_invalid_packet = 0 ds_cksum_err = 0 ds_no_fmd = 0 ds_invalid_vnid = 0 ds_frag_err = 0 ds_invalid_source = 0 ds_mcast_df_bit = 0 ds_l2_no_route = 0 ds_vlan_fwd_tx = 0 ds_vlan_fwd_enq = 0 ds_drop_new_flow = 0 ds_flow_evict = 0 ds_trap_original = 0 ds_no_memory = 0 ds_fragment_queue_fail = 0 ds_no_frag_entry = 0 ds_icmp_error = 0 ds_clone_fail = 0 ds_drop_pkts = 0 ], ] ] ] 2025-09-01 Mon 02:08:24:417.386 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011160806912, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: VrouterStats 2025-09-01 Mon 02:08:24:417.431 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011160806912, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:24:417.521 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011160806912, Pid 113882]: SANDESH: Send FAILED: 1756692504416729 [SYS_NOTICE]: VrouterControlStatsTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3993-1 raw_xmpp_stats= [ [ _iter306->first = 10.20.0.14 [ reconnects = 0 in_msgs = 0 out_msgs = 0 ], ] ] raw_ifmap_stats= [ [ _iter307->first = link_delete_parse_errors _iter307->second = 0, _iter307->first = link_update_parse_errors _iter307->second = 0, _iter307->first = node_delete_parse_errors _iter307->second = 0, _iter307->first = node_update_parse_errors _iter307->second = 0, ] ] raw_rt_table_size= [ [ _iter308->first = default-domain:default-project:ip-fabric:__default__ [ inet4_unicast = 4 inet4_multicast = 0 evpn = 0 bridge = 2 inet6_unicast = 2 ], _iter308->first = default-domain:default-project:ip-fabric:ip-fabric [ inet4_unicast = 2 inet4_multicast = 0 evpn = 1 bridge = 2 inet6_unicast = 0 ], ] ] ] 2025-09-01 Mon 02:08:24:417.554 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011160806912, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: VrouterControlStatsTrace 2025-09-01 Mon 02:08:25:421.831 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011139814912, Pid 113882]: Processing scm::EvIdleHoldTimerExpired in state Idle 2025-09-01 Mon 02:08:25:422.208 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011139814912, Pid 113882]: Connect : Start Connect timer 10.0.0.56:8086 2025-09-01 Mon 02:08:25:422.258 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011171994240, Pid 113882]: -(-1) OnSessionEvent TCP Connect Failed 2025-09-01 Mon 02:08:25:422.329 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011171994240, Pid 113882]: -(-1):EvTcpConnectFail 2025-09-01 Mon 02:08:25:422.354 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011139814912, Pid 113882]: Processing scm::EvSandeshSend in state Connect 2025-09-01 Mon 02:08:25:422.400 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011139814912, Pid 113882]: SANDESH: Send FAILED: 1756692505421944 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-3993-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.14 server_addrs= [ [ (*_iter6) = 10.20.0.14:5269, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.14 server_addrs= [ [ (*_iter6) = 10.20.0.14:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.56:8086, ] ] status = Initializing description = Idle : EvIdleHoldTimerExpired -> Connect ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.14, XMPP:dns-server:10.20.0.14, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:08:25:422.414 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011139814912, Pid 113882]: Wrong state: Connect for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:08:25:422.427 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011139814912, Pid 113882]: Processing scm::EvSandeshSend in state Connect 2025-09-01 Mon 02:08:25:422.682 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011139814912, Pid 113882]: SANDESH: Send FAILED: 1756692505422324 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3993-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Connect successful_connections = 0 pid = 113882 http_port = 8085 start_time = 1756692234402667 collector_name = collector_ip = 10.0.0.56:8086 collector_list= [ [ (*_iter40) = 10.0.0.56:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ComputeCpuStateTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 4 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 31 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 46 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 69 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterControlStatsTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 9 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 13 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 108 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 28 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 100, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 216, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:08:25:422.724 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011139814912, Pid 113882]: Wrong state: Connect for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:08:25:422.741 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011139814912, Pid 113882]: Processing scm::EvTcpConnectFail in state Connect 2025-09-01 Mon 02:08:25:422.751 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011139814912, Pid 113882]: Connect : EvTcpConnectFail 2025-09-01 Mon 02:08:25:422.817 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011139814912, Pid 113882]: TCP [SYS_ERR]: TcpSessionMessageLog: Session -(-1) > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:08:25:422.860 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011139814912, Pid 113882]: Idle 2025-09-01 Mon 02:08:25:422.948 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011139814912, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:25:422.980 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011139814912, Pid 113882]: SANDESH: Send FAILED: 1756692505422785 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-3993-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.14 server_addrs= [ [ (*_iter6) = 10.20.0.14:5269, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.14 server_addrs= [ [ (*_iter6) = 10.20.0.14:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.14, XMPP:dns-server:10.20.0.14, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:08:25:422.990 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011139814912, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:08:25:423.001 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011139814912, Pid 113882]: Processing scm::EvTcpDeleteSession in state Idle 2025-09-01 Mon 02:08:25:423.039 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011139814912, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:25:423.309 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011139814912, Pid 113882]: SANDESH: Send FAILED: 1756692505422929 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3993-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Idle successful_connections = 0 pid = 113882 http_port = 8085 start_time = 1756692234402667 collector_name = collector_ip = 10.0.0.56:8086 collector_list= [ [ (*_iter40) = 10.0.0.56:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ComputeCpuStateTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 4 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 32 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 47 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 70 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterControlStatsTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 9 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 13 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 108 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 28 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 101, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 218, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:08:25:423.339 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011139814912, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:08:27:774.057 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011171994240, Pid 113882]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.0.0.56:8085::10.0.0.56:43006 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 255 2025-09-01 Mon 02:08:34:334.650 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011171994240, Pid 113882]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.20.0.14:8085::10.20.0.14:55732 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 255 2025-09-01 Mon 02:08:34:350.898 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011171994240, Pid 113882]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.20.0.14:8085::10.20.0.14:55746 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 255 2025-09-01 Mon 02:08:40:649.948 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011171994240, Pid 113882]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.0.0.56:8085::10.0.0.56:47588 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 255 2025-09-01 Mon 02:08:44:897.124 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011148211712, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:44:897.228 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011148211712, Pid 113882]: SANDESH: Send FAILED: 1756692524897085 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.14 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:44:897.245 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011148211712, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:44:897.378 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011171994240, Pid 113882]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.14 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:08:44:897.385 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011156608512, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:44:897.457 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011148211712, Pid 113882]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:08:44:897.502 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011156608512, Pid 113882]: SANDESH: Send FAILED: 1756692524897349 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.14 state = Connect event = Connect Event local_port = 37345 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:44:897.534 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011156608512, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:44:897.555 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011156608512, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:44:897.572 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011156608512, Pid 113882]: SANDESH: Send FAILED: 1756692524897544 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.14 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:44:897.579 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011156608512, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:44:897.587 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011156608512, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:44:897.598 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011156608512, Pid 113882]: SANDESH: Send FAILED: 1756692524897554 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.14 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:44:897.605 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011156608512, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:46:651.078 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011156608512, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:46:651.135 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011156608512, Pid 113882]: SANDESH: Send FAILED: 1756692526651038 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.14 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:46:651.145 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011156608512, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:46:651.477 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011148211712, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:46:651.582 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011148211712, Pid 113882]: SANDESH: Send FAILED: 1756692526651404 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.14 state = Connect event = Connect Event local_port = 41577 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:46:651.609 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011148211712, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:46:651.485 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011171994240, Pid 113882]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.14 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:08:46:651.832 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:08:46:652.029 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011152410112, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:46:652.110 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011152410112, Pid 113882]: SANDESH: Send FAILED: 1756692526651939 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.14 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:46:652.135 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011152410112, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:46:652.168 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011152410112, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:46:652.217 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011152410112, Pid 113882]: SANDESH: Send FAILED: 1756692526651993 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.14 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:46:652.236 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011152410112, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:53:834.225 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011171994240, Pid 113882]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.0.0.56:8085::10.0.0.56:42924 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 255 2025-09-01 Mon 02:08:54:410.427 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011148211712, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:54:410.618 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011148211712, Pid 113882]: SANDESH: Send FAILED: 1756692534410055 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3993-1:Compute:contrail-vrouter-agent:0 sm_queue_count = 0 sm_stats= [ ev_stats= [ [ [ event = scm::EvCollectorUpdate enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvIdleHoldTimerExpired enqueues = 13 dequeues = 13 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvSandeshSend enqueues = 228 dequeues = 228 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvStart enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpConnectFail enqueues = 14 dequeues = 14 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpDeleteSession enqueues = 14 dequeues = 14 enqueue_fails = 0 dequeue_fails = 0 ], [ event = enqueues = 271 dequeues = 271 enqueue_fails = 0 dequeue_fails = 0 ], ] ] state = Idle state_since = 1756692505422859 last_state = Idle last_event = scm::EvSandeshSend last_event_at = 1756692526652164 ] max_sm_queue_count = 5 ] 2025-09-01 Mon 02:08:54:410.676 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011148211712, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:08:54:410.703 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011148211712, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:54:411.099 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011148211712, Pid 113882]: SANDESH: Send FAILED: 1756692534410322 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3993-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Idle successful_connections = 0 pid = 113882 http_port = 8085 start_time = 1756692234402667 collector_name = collector_ip = 10.0.0.56:8086 collector_list= [ [ (*_iter40) = 10.0.0.56:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ComputeCpuStateTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 4 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 33 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 48 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 77 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterControlStatsTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 9 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 13 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 116 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 30 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 110, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 228, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:08:54:411.146 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011148211712, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:08:54:417.105 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:54:417.201 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: SANDESH: Send FAILED: 1756692534417010 [SYS_NOTICE]: VrouterStats: data= [ name = cn-jenkins-deploy-platform-ansible-os-3993-1 cpu_info= [ num_cpu = 8 sys_mem_info= [ total = 32 used = 0 free = 0 buffers = 0 cached = 3882065592 node_type = ] meminfo= [ virt = 32598 peakvirt = 0 res = 0 ] cpuload= [ one_min_avg = 0.1575 five_min_avg = 0.2425 fifteen_min_avg = 0.245 ] cpu_share = 0.443676 ] ] 2025-09-01 Mon 02:08:54:417.215 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: VrouterStats 2025-09-01 Mon 02:08:54:417.227 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:54:417.246 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: SANDESH: Send FAILED: 1756692534417072 [SYS_NOTICE]: ComputeCpuStateTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3993-1 cpu_info= [ [ [ mem_virt = 32598 cpu_share = 0.443676 used_sys_mem = 412901704 one_min_cpuload = 0.1575 mem_res = 0 ], ] ] ] 2025-09-01 Mon 02:08:54:417.255 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: ComputeCpuStateTrace 2025-09-01 Mon 02:08:54:417.265 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:54:417.405 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: SANDESH: Send FAILED: 1756692534417121 [SYS_NOTICE]: VrouterStats: data= [ name = cn-jenkins-deploy-platform-ansible-os-3993-1 in_tpkts = 156 in_bytes = 6720 out_tpkts = 157 out_bytes = 6790 exception_packets = 12 exception_packets_dropped = 12 raw_vhost_stats= [ name = vhost0 in_pkts = 156 in_bytes = 6720 out_pkts = 157 out_bytes = 6790 speed = -1 duplexity = -1 ] raw_vhost_drop_stats= [ ds_discard = 0 ds_pull = 0 ds_invalid_if = 0 ds_invalid_arp = 0 ds_trap_no_if = 0 ds_nowhere_to_go = 0 ds_flow_queue_limit_exceeded = 0 ds_flow_no_memory = 0 ds_flow_invalid_protocol = 0 ds_flow_nat_no_rflow = 0 ds_flow_action_drop = 0 ds_flow_action_invalid = 0 ds_flow_unusable = 0 ds_flow_table_full = 0 ds_interface_tx_discard = 0 ds_interface_drop = 0 ds_duplicated = 0 ds_push = 0 ds_ttl_exceeded = 0 ds_invalid_nh = 0 ds_invalid_label = 0 ds_invalid_protocol = 0 ds_interface_rx_discard = 0 ds_invalid_mcast_source = 0 ds_head_alloc_fail = 0 ds_pcow_fail = 0 ds_mcast_clone_fail = 0 ds_rewrite_fail = 0 ds_misc = 0 ds_invalid_packet = 0 ds_cksum_err = 0 ds_no_fmd = 0 ds_invalid_vnid = 0 ds_frag_err = 0 ds_invalid_source = 0 ds_mcast_df_bit = 0 ds_l2_no_route = 0 ds_vlan_fwd_tx = 0 ds_vlan_fwd_enq = 0 ds_drop_new_flow = 0 ds_flow_evict = 0 ds_trap_original = 0 ds_no_memory = 0 ds_fragment_queue_fail = 0 ds_no_frag_entry = 0 ds_icmp_error = 0 ds_clone_fail = 0 ds_drop_pkts = 0 ] raw_drop_stats= [ ds_discard = 0 ds_pull = 0 ds_invalid_if = 0 ds_invalid_arp = 0 ds_trap_no_if = 0 ds_nowhere_to_go = 0 ds_flow_queue_limit_exceeded = 0 ds_flow_no_memory = 0 ds_flow_invalid_protocol = 0 ds_flow_nat_no_rflow = 0 ds_flow_action_drop = 0 ds_flow_action_invalid = 0 ds_flow_unusable = 0 ds_flow_table_full = 0 ds_interface_tx_discard = 0 ds_interface_drop = 0 ds_duplicated = 0 ds_push = 0 ds_ttl_exceeded = 0 ds_invalid_nh = 0 ds_invalid_label = 0 ds_invalid_protocol = 0 ds_interface_rx_discard = 0 ds_invalid_mcast_source = 0 ds_head_alloc_fail = 0 ds_pcow_fail = 0 ds_mcast_clone_fail = 0 ds_rewrite_fail = 0 ds_misc = 0 ds_invalid_packet = 0 ds_cksum_err = 0 ds_no_fmd = 0 ds_invalid_vnid = 0 ds_frag_err = 0 ds_invalid_source = 0 ds_mcast_df_bit = 0 ds_l2_no_route = 0 ds_vlan_fwd_tx = 0 ds_vlan_fwd_enq = 0 ds_drop_new_flow = 0 ds_flow_evict = 0 ds_trap_original = 0 ds_no_memory = 0 ds_fragment_queue_fail = 0 ds_no_frag_entry = 0 ds_icmp_error = 0 ds_clone_fail = 0 ds_drop_pkts = 0 ] flow_rate= [ added_flows = 0 max_flow_adds_per_second = 0 min_flow_adds_per_second = 0 deleted_flows = 0 max_flow_deletes_per_second = 0 min_flow_deletes_per_second = 0 active_flows = 0 hold_flows = 0 ] raw_phy_if_stats= [ [ _iter262->first = ens6 [ in_pkts = 7 in_bytes = 490 out_pkts = 156 out_bytes = 6720 ], ] ] raw_phy_if_drop_stats= [ [ _iter263->first = ens6 [ ds_discard = 0 ds_pull = 0 ds_invalid_if = 0 ds_invalid_arp = 0 ds_trap_no_if = 0 ds_nowhere_to_go = 0 ds_flow_queue_limit_exceeded = 0 ds_flow_no_memory = 0 ds_flow_invalid_protocol = 0 ds_flow_nat_no_rflow = 0 ds_flow_action_drop = 0 ds_flow_action_invalid = 0 ds_flow_unusable = 0 ds_flow_table_full = 0 ds_interface_tx_discard = 0 ds_interface_drop = 0 ds_duplicated = 0 ds_push = 0 ds_ttl_exceeded = 0 ds_invalid_nh = 0 ds_invalid_label = 0 ds_invalid_protocol = 0 ds_interface_rx_discard = 0 ds_invalid_mcast_source = 0 ds_head_alloc_fail = 0 ds_pcow_fail = 0 ds_mcast_clone_fail = 0 ds_rewrite_fail = 0 ds_misc = 0 ds_invalid_packet = 0 ds_cksum_err = 0 ds_no_fmd = 0 ds_invalid_vnid = 0 ds_frag_err = 0 ds_invalid_source = 0 ds_mcast_df_bit = 0 ds_l2_no_route = 0 ds_vlan_fwd_tx = 0 ds_vlan_fwd_enq = 0 ds_drop_new_flow = 0 ds_flow_evict = 0 ds_trap_original = 0 ds_no_memory = 0 ds_fragment_queue_fail = 0 ds_no_frag_entry = 0 ds_icmp_error = 0 ds_clone_fail = 0 ds_drop_pkts = 0 ], ] ] ] 2025-09-01 Mon 02:08:54:417.432 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: VrouterStats 2025-09-01 Mon 02:08:54:417.442 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:54:417.474 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: SANDESH: Send FAILED: 1756692534417201 [SYS_NOTICE]: VrouterControlStatsTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3993-1 raw_xmpp_stats= [ [ _iter306->first = 10.20.0.14 [ reconnects = 0 in_msgs = 0 out_msgs = 0 ], ] ] raw_ifmap_stats= [ [ _iter307->first = link_delete_parse_errors _iter307->second = 0, _iter307->first = link_update_parse_errors _iter307->second = 0, _iter307->first = node_delete_parse_errors _iter307->second = 0, _iter307->first = node_update_parse_errors _iter307->second = 0, ] ] raw_rt_table_size= [ [ _iter308->first = default-domain:default-project:ip-fabric:__default__ [ inet4_unicast = 4 inet4_multicast = 0 evpn = 0 bridge = 2 inet6_unicast = 2 ], _iter308->first = default-domain:default-project:ip-fabric:ip-fabric [ inet4_unicast = 2 inet4_multicast = 0 evpn = 1 bridge = 2 inet6_unicast = 0 ], ] ] ] 2025-09-01 Mon 02:08:54:417.485 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: VrouterControlStatsTrace 2025-09-01 Mon 02:08:55:423.141 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011139814912, Pid 113882]: Processing scm::EvIdleHoldTimerExpired in state Idle 2025-09-01 Mon 02:08:55:423.509 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011139814912, Pid 113882]: Connect : Start Connect timer 10.0.0.56:8086 2025-09-01 Mon 02:08:55:423.585 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011171994240, Pid 113882]: -(-1) OnSessionEvent TCP Connect Failed 2025-09-01 Mon 02:08:55:423.631 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011139814912, Pid 113882]: Processing scm::EvSandeshSend in state Connect 2025-09-01 Mon 02:08:55:423.674 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011171994240, Pid 113882]: -(-1):EvTcpConnectFail 2025-09-01 Mon 02:08:55:423.712 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011139814912, Pid 113882]: SANDESH: Send FAILED: 1756692535423274 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-3993-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.14 server_addrs= [ [ (*_iter6) = 10.20.0.14:5269, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.14 server_addrs= [ [ (*_iter6) = 10.20.0.14:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.56:8086, ] ] status = Initializing description = Idle : EvIdleHoldTimerExpired -> Connect ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.14, XMPP:dns-server:10.20.0.14, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:08:55:423.756 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011139814912, Pid 113882]: Wrong state: Connect for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:08:55:423.769 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011139814912, Pid 113882]: Processing scm::EvSandeshSend in state Connect 2025-09-01 Mon 02:08:55:424.002 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011139814912, Pid 113882]: SANDESH: Send FAILED: 1756692535423611 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3993-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Connect successful_connections = 0 pid = 113882 http_port = 8085 start_time = 1756692234402667 collector_name = collector_ip = 10.0.0.56:8086 collector_list= [ [ (*_iter40) = 10.0.0.56:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ComputeCpuStateTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 5 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 33 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 50 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 77 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterControlStatsTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 10 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 15 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 116 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 30 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 110, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 234, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:08:55:424.036 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011139814912, Pid 113882]: Wrong state: Connect for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:08:55:424.057 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011139814912, Pid 113882]: Processing scm::EvTcpConnectFail in state Connect 2025-09-01 Mon 02:08:55:424.066 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011139814912, Pid 113882]: Connect : EvTcpConnectFail 2025-09-01 Mon 02:08:55:424.123 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011139814912, Pid 113882]: TCP [SYS_ERR]: TcpSessionMessageLog: Session -(-1) > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:08:55:424.166 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011139814912, Pid 113882]: Idle 2025-09-01 Mon 02:08:55:424.256 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011139814912, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:55:424.281 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011139814912, Pid 113882]: SANDESH: Send FAILED: 1756692535424096 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-3993-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.14 server_addrs= [ [ (*_iter6) = 10.20.0.14:5269, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.14 server_addrs= [ [ (*_iter6) = 10.20.0.14:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.14, XMPP:dns-server:10.20.0.14, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:08:55:424.291 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011139814912, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:08:55:424.301 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011139814912, Pid 113882]: Processing scm::EvTcpDeleteSession in state Idle 2025-09-01 Mon 02:08:55:424.333 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011139814912, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:55:424.584 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011139814912, Pid 113882]: SANDESH: Send FAILED: 1756692535424242 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3993-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Idle successful_connections = 0 pid = 113882 http_port = 8085 start_time = 1756692234402667 collector_name = collector_ip = 10.0.0.56:8086 collector_list= [ [ (*_iter40) = 10.0.0.56:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ComputeCpuStateTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 5 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 34 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 51 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 78 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterControlStatsTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 10 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 15 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 116 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 30 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 111, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 236, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:08:55:424.610 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011139814912, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:09:07:011.004 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011171994240, Pid 113882]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.0.0.56:8085::10.0.0.56:54150 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 255 2025-09-01 Mon 02:09:16:487.954 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011148211712, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:16:488.079 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011148211712, Pid 113882]: SANDESH: Send FAILED: 1756692556487887 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.14 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:16:488.107 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011148211712, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:16:488.416 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011171994240, Pid 113882]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.14 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:09:16:488.656 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011148211712, Pid 113882]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:09:16:488.693 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011135616512, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:16:488.812 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011135616512, Pid 113882]: SANDESH: Send FAILED: 1756692556488352 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.14 state = Connect event = Connect Event local_port = 33531 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:16:488.854 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011135616512, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:16:488.892 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011135616512, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:16:488.941 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011135616512, Pid 113882]: SANDESH: Send FAILED: 1756692556488729 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.14 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:16:488.968 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011135616512, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:16:489.002 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011135616512, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:16:489.047 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011135616512, Pid 113882]: SANDESH: Send FAILED: 1756692556488742 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.14 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:16:489.075 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011135616512, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:17:162.354 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011139814912, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:17:162.456 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011139814912, Pid 113882]: SANDESH: Send FAILED: 1756692557162271 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.14 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:17:162.479 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011139814912, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:17:162.697 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011135616512, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:17:162.728 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011171994240, Pid 113882]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.14 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:09:17:162.814 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011135616512, Pid 113882]: SANDESH: Send FAILED: 1756692557162626 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.14 state = Connect event = Connect Event local_port = 41449 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:17:162.915 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011135616512, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:17:163.055 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011148211712, Pid 113882]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:09:17:163.264 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011152410112, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:17:163.295 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011152410112, Pid 113882]: SANDESH: Send FAILED: 1756692557163163 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.14 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:17:163.307 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011152410112, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:17:163.320 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011152410112, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:17:163.337 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011152410112, Pid 113882]: SANDESH: Send FAILED: 1756692557163247 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.14 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:17:163.347 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011152410112, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:20:095.370 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011171994240, Pid 113882]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.0.0.56:8085::10.0.0.56:47378 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 255 2025-09-01 Mon 02:09:24:410.785 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011148211712, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:24:410.911 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011148211712, Pid 113882]: SANDESH: Send FAILED: 1756692564410561 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3993-1:Compute:contrail-vrouter-agent:0 sm_queue_count = 0 sm_stats= [ ev_stats= [ [ [ event = scm::EvCollectorUpdate enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvIdleHoldTimerExpired enqueues = 14 dequeues = 14 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvSandeshSend enqueues = 246 dequeues = 246 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvStart enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpConnectFail enqueues = 15 dequeues = 15 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpDeleteSession enqueues = 15 dequeues = 15 enqueue_fails = 0 dequeue_fails = 0 ], [ event = enqueues = 292 dequeues = 292 enqueue_fails = 0 dequeue_fails = 0 ], ] ] state = Idle state_since = 1756692535424165 last_state = Idle last_event = scm::EvSandeshSend last_event_at = 1756692557163318 ] max_sm_queue_count = 5 ] 2025-09-01 Mon 02:09:24:410.932 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011148211712, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:09:24:410.949 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011148211712, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:24:411.123 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011148211712, Pid 113882]: SANDESH: Send FAILED: 1756692564410703 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3993-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Idle successful_connections = 0 pid = 113882 http_port = 8085 start_time = 1756692234402667 collector_name = collector_ip = 10.0.0.56:8086 collector_list= [ [ (*_iter40) = 10.0.0.56:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ComputeCpuStateTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 5 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 35 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 52 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 82 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterControlStatsTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 10 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 15 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 124 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 32 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 117, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 246, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:09:24:411.155 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011148211712, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:09:24:417.492 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011156608512, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:24:417.667 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011156608512, Pid 113882]: SANDESH: Send FAILED: 1756692564417455 [SYS_NOTICE]: VrouterStats: data= [ name = cn-jenkins-deploy-platform-ansible-os-3993-1 in_tpkts = 159 in_bytes = 6846 out_tpkts = 160 out_bytes = 6916 raw_vhost_stats= [ name = vhost0 in_pkts = 159 in_bytes = 6846 out_pkts = 160 out_bytes = 6916 speed = -1 duplexity = -1 ] raw_vhost_drop_stats= [ ds_discard = 0 ds_pull = 0 ds_invalid_if = 0 ds_invalid_arp = 0 ds_trap_no_if = 0 ds_nowhere_to_go = 0 ds_flow_queue_limit_exceeded = 0 ds_flow_no_memory = 0 ds_flow_invalid_protocol = 0 ds_flow_nat_no_rflow = 0 ds_flow_action_drop = 0 ds_flow_action_invalid = 0 ds_flow_unusable = 0 ds_flow_table_full = 0 ds_interface_tx_discard = 0 ds_interface_drop = 0 ds_duplicated = 0 ds_push = 0 ds_ttl_exceeded = 0 ds_invalid_nh = 0 ds_invalid_label = 0 ds_invalid_protocol = 0 ds_interface_rx_discard = 0 ds_invalid_mcast_source = 0 ds_head_alloc_fail = 0 ds_pcow_fail = 0 ds_mcast_clone_fail = 0 ds_rewrite_fail = 0 ds_misc = 0 ds_invalid_packet = 0 ds_cksum_err = 0 ds_no_fmd = 0 ds_invalid_vnid = 0 ds_frag_err = 0 ds_invalid_source = 0 ds_mcast_df_bit = 0 ds_l2_no_route = 0 ds_vlan_fwd_tx = 0 ds_vlan_fwd_enq = 0 ds_drop_new_flow = 0 ds_flow_evict = 0 ds_trap_original = 0 ds_no_memory = 0 ds_fragment_queue_fail = 0 ds_no_frag_entry = 0 ds_icmp_error = 0 ds_clone_fail = 0 ds_drop_pkts = 0 ] phy_if_5min_usage= [ [ [ name = ens6 in_bandwidth_usage = 9 out_bandwidth_usage = 157 ], ] ] raw_drop_stats= [ ds_discard = 0 ds_pull = 0 ds_invalid_if = 0 ds_invalid_arp = 0 ds_trap_no_if = 0 ds_nowhere_to_go = 0 ds_flow_queue_limit_exceeded = 0 ds_flow_no_memory = 0 ds_flow_invalid_protocol = 0 ds_flow_nat_no_rflow = 0 ds_flow_action_drop = 0 ds_flow_action_invalid = 0 ds_flow_unusable = 0 ds_flow_table_full = 0 ds_interface_tx_discard = 0 ds_interface_drop = 0 ds_duplicated = 0 ds_push = 0 ds_ttl_exceeded = 0 ds_invalid_nh = 0 ds_invalid_label = 0 ds_invalid_protocol = 0 ds_interface_rx_discard = 0 ds_invalid_mcast_source = 0 ds_head_alloc_fail = 0 ds_pcow_fail = 0 ds_mcast_clone_fail = 0 ds_rewrite_fail = 0 ds_misc = 0 ds_invalid_packet = 0 ds_cksum_err = 0 ds_no_fmd = 0 ds_invalid_vnid = 0 ds_frag_err = 0 ds_invalid_source = 0 ds_mcast_df_bit = 0 ds_l2_no_route = 0 ds_vlan_fwd_tx = 0 ds_vlan_fwd_enq = 0 ds_drop_new_flow = 0 ds_flow_evict = 0 ds_trap_original = 0 ds_no_memory = 0 ds_fragment_queue_fail = 0 ds_no_frag_entry = 0 ds_icmp_error = 0 ds_clone_fail = 0 ds_drop_pkts = 0 ] total_in_bandwidth_utilization = 0 total_out_bandwidth_utilization = 4.8247e-16 flow_rate= [ added_flows = 0 max_flow_adds_per_second = 0 min_flow_adds_per_second = 0 deleted_flows = 0 max_flow_deletes_per_second = 0 min_flow_deletes_per_second = 0 active_flows = 0 hold_flows = 0 ] phy_band_in_bps= [ [ _iter260->first = ens6 _iter260->second = 0, ] ] phy_band_out_bps= [ [ _iter261->first = ens6 _iter261->second = 89, ] ] raw_phy_if_stats= [ [ _iter262->first = ens6 [ in_pkts = 7 in_bytes = 490 out_pkts = 159 out_bytes = 6846 ], ] ] raw_phy_if_drop_stats= [ [ _iter263->first = ens6 [ ds_discard = 0 ds_pull = 0 ds_invalid_if = 0 ds_invalid_arp = 0 ds_trap_no_if = 0 ds_nowhere_to_go = 0 ds_flow_queue_limit_exceeded = 0 ds_flow_no_memory = 0 ds_flow_invalid_protocol = 0 ds_flow_nat_no_rflow = 0 ds_flow_action_drop = 0 ds_flow_action_invalid = 0 ds_flow_unusable = 0 ds_flow_table_full = 0 ds_interface_tx_discard = 0 ds_interface_drop = 0 ds_duplicated = 0 ds_push = 0 ds_ttl_exceeded = 0 ds_invalid_nh = 0 ds_invalid_label = 0 ds_invalid_protocol = 0 ds_interface_rx_discard = 0 ds_invalid_mcast_source = 0 ds_head_alloc_fail = 0 ds_pcow_fail = 0 ds_mcast_clone_fail = 0 ds_rewrite_fail = 0 ds_misc = 0 ds_invalid_packet = 0 ds_cksum_err = 0 ds_no_fmd = 0 ds_invalid_vnid = 0 ds_frag_err = 0 ds_invalid_source = 0 ds_mcast_df_bit = 0 ds_l2_no_route = 0 ds_vlan_fwd_tx = 0 ds_vlan_fwd_enq = 0 ds_drop_new_flow = 0 ds_flow_evict = 0 ds_trap_original = 0 ds_no_memory = 0 ds_fragment_queue_fail = 0 ds_no_frag_entry = 0 ds_icmp_error = 0 ds_clone_fail = 0 ds_drop_pkts = 0 ], ] ] ] 2025-09-01 Mon 02:09:24:417.695 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011156608512, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: VrouterStats 2025-09-01 Mon 02:09:24:417.706 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011156608512, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:24:417.733 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011156608512, Pid 113882]: SANDESH: Send FAILED: 1756692564417507 [SYS_NOTICE]: VrouterControlStatsTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3993-1 raw_xmpp_stats= [ [ _iter306->first = 10.20.0.14 [ reconnects = 0 in_msgs = 0 out_msgs = 0 ], ] ] raw_ifmap_stats= [ [ _iter307->first = link_delete_parse_errors _iter307->second = 0, _iter307->first = link_update_parse_errors _iter307->second = 0, _iter307->first = node_delete_parse_errors _iter307->second = 0, _iter307->first = node_update_parse_errors _iter307->second = 0, ] ] raw_rt_table_size= [ [ _iter308->first = default-domain:default-project:ip-fabric:__default__ [ inet4_unicast = 4 inet4_multicast = 0 evpn = 0 bridge = 2 inet6_unicast = 2 ], _iter308->first = default-domain:default-project:ip-fabric:ip-fabric [ inet4_unicast = 2 inet4_multicast = 0 evpn = 1 bridge = 2 inet6_unicast = 0 ], ] ] ] 2025-09-01 Mon 02:09:24:417.740 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011156608512, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: VrouterControlStatsTrace 2025-09-01 Mon 02:09:25:424.531 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011148211712, Pid 113882]: Processing scm::EvIdleHoldTimerExpired in state Idle 2025-09-01 Mon 02:09:25:425.087 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011148211712, Pid 113882]: Connect : Start Connect timer 10.0.0.56:8086 2025-09-01 Mon 02:09:25:425.147 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011171994240, Pid 113882]: -(-1) OnSessionEvent TCP Connect Failed 2025-09-01 Mon 02:09:25:425.261 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011171994240, Pid 113882]: -(-1):EvTcpConnectFail 2025-09-01 Mon 02:09:25:425.401 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011148211712, Pid 113882]: Processing scm::EvSandeshSend in state Connect 2025-09-01 Mon 02:09:25:425.504 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011148211712, Pid 113882]: SANDESH: Send FAILED: 1756692565424698 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-3993-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.14 server_addrs= [ [ (*_iter6) = 10.20.0.14:5269, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.14 server_addrs= [ [ (*_iter6) = 10.20.0.14:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.56:8086, ] ] status = Initializing description = Idle : EvIdleHoldTimerExpired -> Connect ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.14, XMPP:dns-server:10.20.0.14, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:09:25:425.565 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011148211712, Pid 113882]: Wrong state: Connect for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:09:25:425.601 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011148211712, Pid 113882]: Processing scm::EvTcpConnectFail in state Connect 2025-09-01 Mon 02:09:25:425.623 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011148211712, Pid 113882]: Connect : EvTcpConnectFail 2025-09-01 Mon 02:09:25:425.738 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011148211712, Pid 113882]: TCP [SYS_ERR]: TcpSessionMessageLog: Session -(-1) > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:09:25:425.806 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011148211712, Pid 113882]: Idle 2025-09-01 Mon 02:09:25:425.969 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011148211712, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:25:426.474 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011148211712, Pid 113882]: SANDESH: Send FAILED: 1756692565425340 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3993-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Connect successful_connections = 0 pid = 113882 http_port = 8085 start_time = 1756692234402667 collector_name = collector_ip = 10.0.0.56:8086 collector_list= [ [ (*_iter40) = 10.0.0.56:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ComputeCpuStateTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 5 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 35 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 54 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 82 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterControlStatsTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 11 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 16 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 124 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 32 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 117, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 250, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:09:25:426.555 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011148211712, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:09:25:426.588 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011148211712, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:25:426.637 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011148211712, Pid 113882]: SANDESH: Send FAILED: 1756692565425682 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-3993-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.14 server_addrs= [ [ (*_iter6) = 10.20.0.14:5269, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.14 server_addrs= [ [ (*_iter6) = 10.20.0.14:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.14, XMPP:dns-server:10.20.0.14, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:09:25:426.656 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011148211712, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:09:25:426.681 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011148211712, Pid 113882]: Processing scm::EvTcpDeleteSession in state Idle 2025-09-01 Mon 02:09:25:426.753 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011148211712, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:25:427.227 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011148211712, Pid 113882]: SANDESH: Send FAILED: 1756692565425929 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3993-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Idle successful_connections = 0 pid = 113882 http_port = 8085 start_time = 1756692234402667 collector_name = collector_ip = 10.0.0.56:8086 collector_list= [ [ (*_iter40) = 10.0.0.56:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ComputeCpuStateTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 5 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 36 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 54 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 83 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterControlStatsTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 11 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 16 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 124 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 32 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 118, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 251, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:09:25:427.556 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011148211712, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:09:33:366.809 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011171994240, Pid 113882]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.0.0.56:8085::10.0.0.56:35946 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 255 2025-09-01 Mon 02:09:35:297.878 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011171994240, Pid 113882]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.20.0.14:8085::10.20.0.14:59520 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 255 2025-09-01 Mon 02:09:35:321.098 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011171994240, Pid 113882]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.20.0.14:8085::10.20.0.14:59530 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 255 2025-09-01 Mon 02:09:46:263.899 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011171994240, Pid 113882]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.0.0.56:8085::10.0.0.56:59040 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 255 2025-09-01 Mon 02:09:46:323.569 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011148211712, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:46:323.642 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011148211712, Pid 113882]: SANDESH: Send FAILED: 1756692586323524 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.14 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:46:323.659 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011148211712, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:46:323.872 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011171994240, Pid 113882]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.14 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:09:46:323.959 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011152410112, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:46:323.962 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:09:46:324.075 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011152410112, Pid 113882]: SANDESH: Send FAILED: 1756692586323850 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.14 state = Connect event = Connect Event local_port = 44811 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:46:324.213 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011152410112, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:46:324.244 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011152410112, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:46:324.277 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011152410112, Pid 113882]: SANDESH: Send FAILED: 1756692586324165 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.14 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:46:324.324 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011152410112, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:46:324.345 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011152410112, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:46:324.372 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011152410112, Pid 113882]: SANDESH: Send FAILED: 1756692586324206 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.14 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:46:324.390 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011152410112, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:46:999.171 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011139814912, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:46:999.322 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011139814912, Pid 113882]: SANDESH: Send FAILED: 1756692586999116 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.14 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:46:999.352 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011139814912, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:46:999.565 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:46:999.589 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011171994240, Pid 113882]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.14 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:09:46:999.712 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: SANDESH: Send FAILED: 1756692586999499 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.14 state = Connect event = Connect Event local_port = 42979 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:46:999.798 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:46:999.897 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:09:46:999.991 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011152410112, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:47:000.029 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011152410112, Pid 113882]: SANDESH: Send FAILED: 1756692586999969 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.14 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:47:000.041 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011152410112, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:47:000.051 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011152410112, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:47:000.065 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011152410112, Pid 113882]: SANDESH: Send FAILED: 1756692586999984 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.14 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:47:000.091 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011152410112, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:54:411.116 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011152410112, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:54:411.329 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011152410112, Pid 113882]: SANDESH: Send FAILED: 1756692594410884 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3993-1:Compute:contrail-vrouter-agent:0 sm_queue_count = 0 sm_stats= [ ev_stats= [ [ [ event = scm::EvCollectorUpdate enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvIdleHoldTimerExpired enqueues = 15 dequeues = 15 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvSandeshSend enqueues = 262 dequeues = 262 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvStart enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpConnectFail enqueues = 16 dequeues = 16 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpDeleteSession enqueues = 16 dequeues = 16 enqueue_fails = 0 dequeue_fails = 0 ], [ event = enqueues = 311 dequeues = 311 enqueue_fails = 0 dequeue_fails = 0 ], ] ] state = Idle state_since = 1756692565425804 last_state = Idle last_event = scm::EvSandeshSend last_event_at = 1756692587000049 ] max_sm_queue_count = 5 ] 2025-09-01 Mon 02:09:54:411.369 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011152410112, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:09:54:411.401 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011152410112, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:54:411.794 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011152410112, Pid 113882]: SANDESH: Send FAILED: 1756692594411025 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3993-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Idle successful_connections = 0 pid = 113882 http_port = 8085 start_time = 1756692234402667 collector_name = collector_ip = 10.0.0.56:8086 collector_list= [ [ (*_iter40) = 10.0.0.56:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ComputeCpuStateTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 5 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 37 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 56 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 89 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterControlStatsTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 11 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 16 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 132 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 34 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 126, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 262, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:09:54:411.856 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011152410112, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:09:54:417.784 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011139814912, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:54:417.900 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011139814912, Pid 113882]: SANDESH: Send FAILED: 1756692594417739 [SYS_NOTICE]: VrouterStats: data= [ name = cn-jenkins-deploy-platform-ansible-os-3993-1 cpu_info= [ num_cpu = 8 sys_mem_info= [ total = 32 used = 0 free = 0 buffers = 0 cached = 3882065592 node_type = ] meminfo= [ virt = 32598 peakvirt = 0 res = 0 ] cpuload= [ one_min_avg = 0.18 five_min_avg = 0.2325 fifteen_min_avg = 0.24125 ] cpu_share = 0.45625 ] ] 2025-09-01 Mon 02:09:54:417.919 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011139814912, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: VrouterStats 2025-09-01 Mon 02:09:54:417.935 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011139814912, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:54:417.978 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011139814912, Pid 113882]: SANDESH: Send FAILED: 1756692594417763 [SYS_NOTICE]: ComputeCpuStateTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3993-1 cpu_info= [ [ [ mem_virt = 32598 cpu_share = 0.45625 used_sys_mem = 412901704 one_min_cpuload = 0.18 mem_res = 0 ], ] ] ] 2025-09-01 Mon 02:09:54:417.990 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011139814912, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: ComputeCpuStateTrace 2025-09-01 Mon 02:09:54:418.003 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011139814912, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:54:418.212 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011139814912, Pid 113882]: SANDESH: Send FAILED: 1756692594417798 [SYS_NOTICE]: VrouterStats: data= [ name = cn-jenkins-deploy-platform-ansible-os-3993-1 in_tpkts = 162 in_bytes = 6972 out_tpkts = 164 out_bytes = 7112 raw_vhost_stats= [ name = vhost0 in_pkts = 162 in_bytes = 6972 out_pkts = 164 out_bytes = 7112 speed = -1 duplexity = -1 ] raw_vhost_drop_stats= [ ds_discard = 0 ds_pull = 0 ds_invalid_if = 0 ds_invalid_arp = 0 ds_trap_no_if = 0 ds_nowhere_to_go = 0 ds_flow_queue_limit_exceeded = 0 ds_flow_no_memory = 0 ds_flow_invalid_protocol = 0 ds_flow_nat_no_rflow = 0 ds_flow_action_drop = 0 ds_flow_action_invalid = 0 ds_flow_unusable = 0 ds_flow_table_full = 0 ds_interface_tx_discard = 0 ds_interface_drop = 0 ds_duplicated = 0 ds_push = 0 ds_ttl_exceeded = 0 ds_invalid_nh = 0 ds_invalid_label = 0 ds_invalid_protocol = 0 ds_interface_rx_discard = 0 ds_invalid_mcast_source = 0 ds_head_alloc_fail = 0 ds_pcow_fail = 0 ds_mcast_clone_fail = 0 ds_rewrite_fail = 0 ds_misc = 0 ds_invalid_packet = 0 ds_cksum_err = 0 ds_no_fmd = 0 ds_invalid_vnid = 0 ds_frag_err = 0 ds_invalid_source = 0 ds_mcast_df_bit = 0 ds_l2_no_route = 0 ds_vlan_fwd_tx = 0 ds_vlan_fwd_enq = 0 ds_drop_new_flow = 0 ds_flow_evict = 0 ds_trap_original = 0 ds_no_memory = 0 ds_fragment_queue_fail = 0 ds_no_frag_entry = 0 ds_icmp_error = 0 ds_clone_fail = 0 ds_drop_pkts = 0 ] raw_drop_stats= [ ds_discard = 0 ds_pull = 0 ds_invalid_if = 0 ds_invalid_arp = 0 ds_trap_no_if = 0 ds_nowhere_to_go = 0 ds_flow_queue_limit_exceeded = 0 ds_flow_no_memory = 0 ds_flow_invalid_protocol = 0 ds_flow_nat_no_rflow = 0 ds_flow_action_drop = 0 ds_flow_action_invalid = 0 ds_flow_unusable = 0 ds_flow_table_full = 0 ds_interface_tx_discard = 0 ds_interface_drop = 0 ds_duplicated = 0 ds_push = 0 ds_ttl_exceeded = 0 ds_invalid_nh = 0 ds_invalid_label = 0 ds_invalid_protocol = 0 ds_interface_rx_discard = 0 ds_invalid_mcast_source = 0 ds_head_alloc_fail = 0 ds_pcow_fail = 0 ds_mcast_clone_fail = 0 ds_rewrite_fail = 0 ds_misc = 0 ds_invalid_packet = 0 ds_cksum_err = 0 ds_no_fmd = 0 ds_invalid_vnid = 0 ds_frag_err = 0 ds_invalid_source = 0 ds_mcast_df_bit = 0 ds_l2_no_route = 0 ds_vlan_fwd_tx = 0 ds_vlan_fwd_enq = 0 ds_drop_new_flow = 0 ds_flow_evict = 0 ds_trap_original = 0 ds_no_memory = 0 ds_fragment_queue_fail = 0 ds_no_frag_entry = 0 ds_icmp_error = 0 ds_clone_fail = 0 ds_drop_pkts = 0 ] flow_rate= [ added_flows = 0 max_flow_adds_per_second = 0 min_flow_adds_per_second = 0 deleted_flows = 0 max_flow_deletes_per_second = 0 min_flow_deletes_per_second = 0 active_flows = 0 hold_flows = 0 ] raw_phy_if_stats= [ [ _iter262->first = ens6 [ in_pkts = 8 in_bytes = 560 out_pkts = 162 out_bytes = 6972 ], ] ] raw_phy_if_drop_stats= [ [ _iter263->first = ens6 [ ds_discard = 0 ds_pull = 0 ds_invalid_if = 0 ds_invalid_arp = 0 ds_trap_no_if = 0 ds_nowhere_to_go = 0 ds_flow_queue_limit_exceeded = 0 ds_flow_no_memory = 0 ds_flow_invalid_protocol = 0 ds_flow_nat_no_rflow = 0 ds_flow_action_drop = 0 ds_flow_action_invalid = 0 ds_flow_unusable = 0 ds_flow_table_full = 0 ds_interface_tx_discard = 0 ds_interface_drop = 0 ds_duplicated = 0 ds_push = 0 ds_ttl_exceeded = 0 ds_invalid_nh = 0 ds_invalid_label = 0 ds_invalid_protocol = 0 ds_interface_rx_discard = 0 ds_invalid_mcast_source = 0 ds_head_alloc_fail = 0 ds_pcow_fail = 0 ds_mcast_clone_fail = 0 ds_rewrite_fail = 0 ds_misc = 0 ds_invalid_packet = 0 ds_cksum_err = 0 ds_no_fmd = 0 ds_invalid_vnid = 0 ds_frag_err = 0 ds_invalid_source = 0 ds_mcast_df_bit = 0 ds_l2_no_route = 0 ds_vlan_fwd_tx = 0 ds_vlan_fwd_enq = 0 ds_drop_new_flow = 0 ds_flow_evict = 0 ds_trap_original = 0 ds_no_memory = 0 ds_fragment_queue_fail = 0 ds_no_frag_entry = 0 ds_icmp_error = 0 ds_clone_fail = 0 ds_drop_pkts = 0 ], ] ] ] 2025-09-01 Mon 02:09:54:418.244 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011139814912, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: VrouterStats 2025-09-01 Mon 02:09:54:418.283 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011139814912, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:54:418.357 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011139814912, Pid 113882]: SANDESH: Send FAILED: 1756692594417849 [SYS_NOTICE]: VrouterControlStatsTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3993-1 raw_xmpp_stats= [ [ _iter306->first = 10.20.0.14 [ reconnects = 0 in_msgs = 0 out_msgs = 0 ], ] ] raw_ifmap_stats= [ [ _iter307->first = link_delete_parse_errors _iter307->second = 0, _iter307->first = link_update_parse_errors _iter307->second = 0, _iter307->first = node_delete_parse_errors _iter307->second = 0, _iter307->first = node_update_parse_errors _iter307->second = 0, ] ] raw_rt_table_size= [ [ _iter308->first = default-domain:default-project:ip-fabric:__default__ [ inet4_unicast = 4 inet4_multicast = 0 evpn = 0 bridge = 2 inet6_unicast = 2 ], _iter308->first = default-domain:default-project:ip-fabric:ip-fabric [ inet4_unicast = 2 inet4_multicast = 0 evpn = 1 bridge = 2 inet6_unicast = 0 ], ] ] ] 2025-09-01 Mon 02:09:54:418.375 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011139814912, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: VrouterControlStatsTrace 2025-09-01 Mon 02:09:55:425.945 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011156608512, Pid 113882]: Processing scm::EvIdleHoldTimerExpired in state Idle 2025-09-01 Mon 02:09:55:426.216 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011156608512, Pid 113882]: Connect : Start Connect timer 10.0.0.56:8086 2025-09-01 Mon 02:09:55:426.244 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011171994240, Pid 113882]: -(-1) OnSessionEvent TCP Connect Failed 2025-09-01 Mon 02:09:55:426.280 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011171994240, Pid 113882]: -(-1):EvTcpConnectFail 2025-09-01 Mon 02:09:55:426.305 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011156608512, Pid 113882]: Processing scm::EvSandeshSend in state Connect 2025-09-01 Mon 02:09:55:426.329 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011156608512, Pid 113882]: SANDESH: Send FAILED: 1756692595426037 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-3993-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.14 server_addrs= [ [ (*_iter6) = 10.20.0.14:5269, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.14 server_addrs= [ [ (*_iter6) = 10.20.0.14:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.56:8086, ] ] status = Initializing description = Idle : EvIdleHoldTimerExpired -> Connect ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.14, XMPP:dns-server:10.20.0.14, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:09:55:426.336 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011156608512, Pid 113882]: Wrong state: Connect for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:09:55:426.344 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011156608512, Pid 113882]: Processing scm::EvTcpConnectFail in state Connect 2025-09-01 Mon 02:09:55:426.361 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011156608512, Pid 113882]: Connect : EvTcpConnectFail 2025-09-01 Mon 02:09:55:426.399 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011156608512, Pid 113882]: TCP [SYS_ERR]: TcpSessionMessageLog: Session -(-1) > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:09:55:426.438 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011156608512, Pid 113882]: Idle 2025-09-01 Mon 02:09:55:426.487 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011156608512, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:55:426.602 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011156608512, Pid 113882]: SANDESH: Send FAILED: 1756692595426292 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3993-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Connect successful_connections = 0 pid = 113882 http_port = 8085 start_time = 1756692234402667 collector_name = collector_ip = 10.0.0.56:8086 collector_list= [ [ (*_iter40) = 10.0.0.56:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ComputeCpuStateTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 6 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 37 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 58 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 89 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterControlStatsTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 12 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 18 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 132 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 34 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 126, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 268, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:09:55:426.621 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011156608512, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:09:55:426.628 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011156608512, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:55:426.641 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011156608512, Pid 113882]: SANDESH: Send FAILED: 1756692595426379 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-3993-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.14 server_addrs= [ [ (*_iter6) = 10.20.0.14:5269, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.14 server_addrs= [ [ (*_iter6) = 10.20.0.14:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.14, XMPP:dns-server:10.20.0.14, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:09:55:426.647 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011156608512, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:09:55:426.652 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011156608512, Pid 113882]: Processing scm::EvTcpDeleteSession in state Idle 2025-09-01 Mon 02:09:55:426.674 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011156608512, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:55:426.771 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011156608512, Pid 113882]: SANDESH: Send FAILED: 1756692595426470 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3993-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Idle successful_connections = 0 pid = 113882 http_port = 8085 start_time = 1756692234402667 collector_name = collector_ip = 10.0.0.56:8086 collector_list= [ [ (*_iter40) = 10.0.0.56:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ComputeCpuStateTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 6 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 38 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 58 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 90 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterControlStatsTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 12 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 18 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 132 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 34 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 127, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 269, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:09:55:426.793 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011156608512, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:09:59:290.516 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011171994240, Pid 113882]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.0.0.56:8085::10.0.0.56:54488 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 255 2025-09-01 Mon 02:10:12:223.961 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011171994240, Pid 113882]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.0.0.56:8085::10.0.0.56:57452 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 255 2025-09-01 Mon 02:10:15:080.371 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011139814912, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:15:080.595 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011139814912, Pid 113882]: SANDESH: Send FAILED: 1756692615080290 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.14 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:15:080.637 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011139814912, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:15:080.675 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011139814912, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:15:080.694 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011171994240, Pid 113882]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.14 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:10:15:080.732 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011139814912, Pid 113882]: SANDESH: Send FAILED: 1756692615080650 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.14 state = Connect event = Connect Event local_port = 34117 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:15:080.809 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011139814912, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:15:080.930 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011139814912, Pid 113882]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:10:15:081.120 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011160806912, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:15:081.225 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011160806912, Pid 113882]: SANDESH: Send FAILED: 1756692615081021 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.14 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:15:081.252 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011160806912, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:15:081.278 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011160806912, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:15:081.306 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011160806912, Pid 113882]: SANDESH: Send FAILED: 1756692615081084 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.14 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:15:081.350 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011160806912, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:17:104.663 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011139814912, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:17:104.740 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011139814912, Pid 113882]: SANDESH: Send FAILED: 1756692617104615 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.14 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:17:104.753 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011139814912, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:17:104.997 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011171994240, Pid 113882]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.14 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:10:17:105.093 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011135616512, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:17:105.103 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011152410112, Pid 113882]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:10:17:105.250 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011135616512, Pid 113882]: SANDESH: Send FAILED: 1756692617104969 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.14 state = Connect event = Connect Event local_port = 38021 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:17:105.314 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011135616512, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:17:105.354 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011135616512, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:17:105.404 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011135616512, Pid 113882]: SANDESH: Send FAILED: 1756692617105314 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.14 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:17:105.477 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011135616512, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:17:105.523 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011135616512, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:17:105.567 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011135616512, Pid 113882]: SANDESH: Send FAILED: 1756692617105328 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.14 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:17:105.594 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011135616512, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:24:411.525 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011160806912, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:24:411.731 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011160806912, Pid 113882]: SANDESH: Send FAILED: 1756692624411269 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3993-1:Compute:contrail-vrouter-agent:0 sm_queue_count = 0 sm_stats= [ ev_stats= [ [ [ event = scm::EvCollectorUpdate enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvIdleHoldTimerExpired enqueues = 16 dequeues = 16 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvSandeshSend enqueues = 280 dequeues = 280 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvStart enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpConnectFail enqueues = 17 dequeues = 17 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpDeleteSession enqueues = 17 dequeues = 17 enqueue_fails = 0 dequeue_fails = 0 ], [ event = enqueues = 332 dequeues = 332 enqueue_fails = 0 dequeue_fails = 0 ], ] ] state = Idle state_since = 1756692595426437 last_state = Idle last_event = scm::EvSandeshSend last_event_at = 1756692617105517 ] max_sm_queue_count = 5 ] 2025-09-01 Mon 02:10:24:411.791 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011160806912, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:10:24:411.831 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011160806912, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:24:412.653 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011160806912, Pid 113882]: SANDESH: Send FAILED: 1756692624411425 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3993-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Idle successful_connections = 0 pid = 113882 http_port = 8085 start_time = 1756692234402667 collector_name = collector_ip = 10.0.0.56:8086 collector_list= [ [ (*_iter40) = 10.0.0.56:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ComputeCpuStateTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 6 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 39 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 60 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 94 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterControlStatsTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 12 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 18 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 140 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 36 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 133, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 280, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:10:24:412.752 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011160806912, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:10:24:418.209 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011152410112, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:24:418.560 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011152410112, Pid 113882]: SANDESH: Send FAILED: 1756692624418150 [SYS_NOTICE]: VrouterStats: data= [ name = cn-jenkins-deploy-platform-ansible-os-3993-1 in_tpkts = 165 in_bytes = 7098 out_tpkts = 167 out_bytes = 7238 raw_vhost_stats= [ name = vhost0 in_pkts = 165 in_bytes = 7098 out_pkts = 167 out_bytes = 7238 speed = -1 duplexity = -1 ] raw_vhost_drop_stats= [ ds_discard = 0 ds_pull = 0 ds_invalid_if = 0 ds_invalid_arp = 0 ds_trap_no_if = 0 ds_nowhere_to_go = 0 ds_flow_queue_limit_exceeded = 0 ds_flow_no_memory = 0 ds_flow_invalid_protocol = 0 ds_flow_nat_no_rflow = 0 ds_flow_action_drop = 0 ds_flow_action_invalid = 0 ds_flow_unusable = 0 ds_flow_table_full = 0 ds_interface_tx_discard = 0 ds_interface_drop = 0 ds_duplicated = 0 ds_push = 0 ds_ttl_exceeded = 0 ds_invalid_nh = 0 ds_invalid_label = 0 ds_invalid_protocol = 0 ds_interface_rx_discard = 0 ds_invalid_mcast_source = 0 ds_head_alloc_fail = 0 ds_pcow_fail = 0 ds_mcast_clone_fail = 0 ds_rewrite_fail = 0 ds_misc = 0 ds_invalid_packet = 0 ds_cksum_err = 0 ds_no_fmd = 0 ds_invalid_vnid = 0 ds_frag_err = 0 ds_invalid_source = 0 ds_mcast_df_bit = 0 ds_l2_no_route = 0 ds_vlan_fwd_tx = 0 ds_vlan_fwd_enq = 0 ds_drop_new_flow = 0 ds_flow_evict = 0 ds_trap_original = 0 ds_no_memory = 0 ds_fragment_queue_fail = 0 ds_no_frag_entry = 0 ds_icmp_error = 0 ds_clone_fail = 0 ds_drop_pkts = 0 ] raw_drop_stats= [ ds_discard = 0 ds_pull = 0 ds_invalid_if = 0 ds_invalid_arp = 0 ds_trap_no_if = 0 ds_nowhere_to_go = 0 ds_flow_queue_limit_exceeded = 0 ds_flow_no_memory = 0 ds_flow_invalid_protocol = 0 ds_flow_nat_no_rflow = 0 ds_flow_action_drop = 0 ds_flow_action_invalid = 0 ds_flow_unusable = 0 ds_flow_table_full = 0 ds_interface_tx_discard = 0 ds_interface_drop = 0 ds_duplicated = 0 ds_push = 0 ds_ttl_exceeded = 0 ds_invalid_nh = 0 ds_invalid_label = 0 ds_invalid_protocol = 0 ds_interface_rx_discard = 0 ds_invalid_mcast_source = 0 ds_head_alloc_fail = 0 ds_pcow_fail = 0 ds_mcast_clone_fail = 0 ds_rewrite_fail = 0 ds_misc = 0 ds_invalid_packet = 0 ds_cksum_err = 0 ds_no_fmd = 0 ds_invalid_vnid = 0 ds_frag_err = 0 ds_invalid_source = 0 ds_mcast_df_bit = 0 ds_l2_no_route = 0 ds_vlan_fwd_tx = 0 ds_vlan_fwd_enq = 0 ds_drop_new_flow = 0 ds_flow_evict = 0 ds_trap_original = 0 ds_no_memory = 0 ds_fragment_queue_fail = 0 ds_no_frag_entry = 0 ds_icmp_error = 0 ds_clone_fail = 0 ds_drop_pkts = 0 ] total_in_bandwidth_utilization = 4.87891e-17 total_out_bandwidth_utilization = 1.78893e-16 flow_rate= [ added_flows = 0 max_flow_adds_per_second = 0 min_flow_adds_per_second = 0 deleted_flows = 0 max_flow_deletes_per_second = 0 min_flow_deletes_per_second = 0 active_flows = 0 hold_flows = 0 ] phy_band_in_bps= [ [ _iter260->first = ens6 _iter260->second = 9, ] ] phy_band_out_bps= [ [ _iter261->first = ens6 _iter261->second = 33, ] ] raw_phy_if_stats= [ [ _iter262->first = ens6 [ in_pkts = 8 in_bytes = 560 out_pkts = 165 out_bytes = 7098 ], ] ] raw_phy_if_drop_stats= [ [ _iter263->first = ens6 [ ds_discard = 0 ds_pull = 0 ds_invalid_if = 0 ds_invalid_arp = 0 ds_trap_no_if = 0 ds_nowhere_to_go = 0 ds_flow_queue_limit_exceeded = 0 ds_flow_no_memory = 0 ds_flow_invalid_protocol = 0 ds_flow_nat_no_rflow = 0 ds_flow_action_drop = 0 ds_flow_action_invalid = 0 ds_flow_unusable = 0 ds_flow_table_full = 0 ds_interface_tx_discard = 0 ds_interface_drop = 0 ds_duplicated = 0 ds_push = 0 ds_ttl_exceeded = 0 ds_invalid_nh = 0 ds_invalid_label = 0 ds_invalid_protocol = 0 ds_interface_rx_discard = 0 ds_invalid_mcast_source = 0 ds_head_alloc_fail = 0 ds_pcow_fail = 0 ds_mcast_clone_fail = 0 ds_rewrite_fail = 0 ds_misc = 0 ds_invalid_packet = 0 ds_cksum_err = 0 ds_no_fmd = 0 ds_invalid_vnid = 0 ds_frag_err = 0 ds_invalid_source = 0 ds_mcast_df_bit = 0 ds_l2_no_route = 0 ds_vlan_fwd_tx = 0 ds_vlan_fwd_enq = 0 ds_drop_new_flow = 0 ds_flow_evict = 0 ds_trap_original = 0 ds_no_memory = 0 ds_fragment_queue_fail = 0 ds_no_frag_entry = 0 ds_icmp_error = 0 ds_clone_fail = 0 ds_drop_pkts = 0 ], ] ] ] 2025-09-01 Mon 02:10:24:418.600 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011152410112, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: VrouterStats 2025-09-01 Mon 02:10:24:418.619 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011152410112, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:24:418.678 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011152410112, Pid 113882]: SANDESH: Send FAILED: 1756692624418258 [SYS_NOTICE]: VrouterControlStatsTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3993-1 raw_xmpp_stats= [ [ _iter306->first = 10.20.0.14 [ reconnects = 0 in_msgs = 0 out_msgs = 0 ], ] ] raw_ifmap_stats= [ [ _iter307->first = link_delete_parse_errors _iter307->second = 0, _iter307->first = link_update_parse_errors _iter307->second = 0, _iter307->first = node_delete_parse_errors _iter307->second = 0, _iter307->first = node_update_parse_errors _iter307->second = 0, ] ] raw_rt_table_size= [ [ _iter308->first = default-domain:default-project:ip-fabric:__default__ [ inet4_unicast = 4 inet4_multicast = 0 evpn = 0 bridge = 2 inet6_unicast = 2 ], _iter308->first = default-domain:default-project:ip-fabric:ip-fabric [ inet4_unicast = 2 inet4_multicast = 0 evpn = 1 bridge = 2 inet6_unicast = 0 ], ] ] ] 2025-09-01 Mon 02:10:24:418.693 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011152410112, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: VrouterControlStatsTrace 2025-09-01 Mon 02:10:25:106.923 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011171994240, Pid 113882]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.0.0.56:8085::10.0.0.56:44520 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 255 2025-09-01 Mon 02:10:25:426.653 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011156608512, Pid 113882]: Processing scm::EvIdleHoldTimerExpired in state Idle 2025-09-01 Mon 02:10:25:426.952 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011156608512, Pid 113882]: Connect : Start Connect timer 10.0.0.56:8086 2025-09-01 Mon 02:10:25:427.021 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011171994240, Pid 113882]: -(-1) OnSessionEvent TCP Connect Failed 2025-09-01 Mon 02:10:25:427.100 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011171994240, Pid 113882]: -(-1):EvTcpConnectFail 2025-09-01 Mon 02:10:25:427.070 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011156608512, Pid 113882]: Processing scm::EvSandeshSend in state Connect 2025-09-01 Mon 02:10:25:427.186 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011156608512, Pid 113882]: SANDESH: Send FAILED: 1756692625426749 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-3993-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.14 server_addrs= [ [ (*_iter6) = 10.20.0.14:5269, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.14 server_addrs= [ [ (*_iter6) = 10.20.0.14:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.56:8086, ] ] status = Initializing description = Idle : EvIdleHoldTimerExpired -> Connect ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.14, XMPP:dns-server:10.20.0.14, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:10:25:427.198 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011156608512, Pid 113882]: Wrong state: Connect for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:10:25:427.210 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011156608512, Pid 113882]: Processing scm::EvSandeshSend in state Connect 2025-09-01 Mon 02:10:25:427.440 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011156608512, Pid 113882]: SANDESH: Send FAILED: 1756692625427045 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3993-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Connect successful_connections = 0 pid = 113882 http_port = 8085 start_time = 1756692234402667 collector_name = collector_ip = 10.0.0.56:8086 collector_list= [ [ (*_iter40) = 10.0.0.56:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ComputeCpuStateTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 6 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 39 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 62 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 95 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterControlStatsTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 13 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 19 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 140 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 36 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 134, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 284, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:10:25:427.470 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011156608512, Pid 113882]: Wrong state: Connect for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:10:25:427.482 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011156608512, Pid 113882]: Processing scm::EvTcpConnectFail in state Connect 2025-09-01 Mon 02:10:25:427.491 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011156608512, Pid 113882]: Connect : EvTcpConnectFail 2025-09-01 Mon 02:10:25:427.543 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011156608512, Pid 113882]: TCP [SYS_ERR]: TcpSessionMessageLog: Session -(-1) > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:10:25:427.575 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011156608512, Pid 113882]: Idle 2025-09-01 Mon 02:10:25:427.643 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011156608512, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:25:427.667 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011156608512, Pid 113882]: SANDESH: Send FAILED: 1756692625427517 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-3993-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.14 server_addrs= [ [ (*_iter6) = 10.20.0.14:5269, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.14 server_addrs= [ [ (*_iter6) = 10.20.0.14:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.14, XMPP:dns-server:10.20.0.14, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:10:25:427.680 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011156608512, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:10:25:427.690 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011156608512, Pid 113882]: Processing scm::EvTcpDeleteSession in state Idle 2025-09-01 Mon 02:10:25:427.722 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011156608512, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:25:427.939 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011156608512, Pid 113882]: SANDESH: Send FAILED: 1756692625427628 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3993-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Idle successful_connections = 0 pid = 113882 http_port = 8085 start_time = 1756692234402667 collector_name = collector_ip = 10.0.0.56:8086 collector_list= [ [ (*_iter40) = 10.0.0.56:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ComputeCpuStateTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 6 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 40 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 63 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 96 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterControlStatsTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 13 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 19 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 140 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 36 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 135, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 286, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:10:25:427.957 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011156608512, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:10:36:309.985 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011171994240, Pid 113882]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.20.0.14:8085::10.20.0.14:58060 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 255 2025-09-01 Mon 02:10:36:327.527 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011171994240, Pid 113882]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.20.0.14:8085::10.20.0.14:58074 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 255 2025-09-01 Mon 02:10:38:451.106 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011171994240, Pid 113882]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.0.0.56:8085::10.0.0.56:60194 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 255 2025-09-01 Mon 02:10:44:781.513 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011139814912, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:44:781.643 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011139814912, Pid 113882]: SANDESH: Send FAILED: 1756692644781426 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.14 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:44:781.672 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011139814912, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:44:781.753 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011171994240, Pid 113882]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.14 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:10:44:781.763 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011152410112, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:44:781.903 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011139814912, Pid 113882]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:10:44:781.907 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011152410112, Pid 113882]: SANDESH: Send FAILED: 1756692644781715 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.14 state = Connect event = Connect Event local_port = 35879 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:44:781.973 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011152410112, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:44:782.005 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011152410112, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:44:782.027 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011152410112, Pid 113882]: SANDESH: Send FAILED: 1756692644781988 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.14 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:44:782.039 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011152410112, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:44:782.053 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011152410112, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:44:782.074 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011152410112, Pid 113882]: SANDESH: Send FAILED: 1756692644782007 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.14 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:44:782.086 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011152410112, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:47:345.634 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011156608512, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:47:345.711 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011156608512, Pid 113882]: SANDESH: Send FAILED: 1756692647345586 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.14 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:47:345.729 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011156608512, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:47:345.949 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011171994240, Pid 113882]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.14 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:10:47:345.987 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011152410112, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:47:346.040 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011156608512, Pid 113882]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:10:47:346.075 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011152410112, Pid 113882]: SANDESH: Send FAILED: 1756692647345920 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.14 state = Connect event = Connect Event local_port = 40107 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:47:346.136 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011152410112, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:47:346.164 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011152410112, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:47:346.239 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011152410112, Pid 113882]: SANDESH: Send FAILED: 1756692647346087 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.14 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:47:346.267 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011152410112, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:47:346.296 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011152410112, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:47:346.338 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011152410112, Pid 113882]: SANDESH: Send FAILED: 1756692647346096 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.14 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:47:346.369 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011152410112, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:51:330.848 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011171994240, Pid 113882]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.0.0.56:8085::10.0.0.56:34320 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 255 2025-09-01 Mon 02:10:54:411.943 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011139814912, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:54:412.154 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011139814912, Pid 113882]: SANDESH: Send FAILED: 1756692654411670 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3993-1:Compute:contrail-vrouter-agent:0 sm_queue_count = 0 sm_stats= [ ev_stats= [ [ [ event = scm::EvCollectorUpdate enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvIdleHoldTimerExpired enqueues = 17 dequeues = 17 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvSandeshSend enqueues = 296 dequeues = 296 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvStart enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpConnectFail enqueues = 18 dequeues = 18 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpDeleteSession enqueues = 18 dequeues = 18 enqueue_fails = 0 dequeue_fails = 0 ], [ event = enqueues = 351 dequeues = 351 enqueue_fails = 0 dequeue_fails = 0 ], ] ] state = Idle state_since = 1756692625427573 last_state = Idle last_event = scm::EvSandeshSend last_event_at = 1756692647346291 ] max_sm_queue_count = 5 ] 2025-09-01 Mon 02:10:54:412.221 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011139814912, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:10:54:412.251 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011139814912, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:54:412.788 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011139814912, Pid 113882]: SANDESH: Send FAILED: 1756692654411812 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3993-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Idle successful_connections = 0 pid = 113882 http_port = 8085 start_time = 1756692234402667 collector_name = collector_ip = 10.0.0.56:8086 collector_list= [ [ (*_iter40) = 10.0.0.56:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ComputeCpuStateTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 6 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 41 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 64 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 102 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterControlStatsTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 13 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 19 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 148 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 38 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 143, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 296, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:10:54:412.871 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011139814912, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:10:54:418.839 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011139814912, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:54:418.952 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011139814912, Pid 113882]: SANDESH: Send FAILED: 1756692654418722 [SYS_NOTICE]: VrouterStats: data= [ name = cn-jenkins-deploy-platform-ansible-os-3993-1 cpu_info= [ num_cpu = 8 sys_mem_info= [ total = 32 used = 0 free = 0 buffers = 0 cached = 3882065592 node_type = ] meminfo= [ virt = 32598 peakvirt = 0 res = 0 ] cpuload= [ one_min_avg = 0.1375 five_min_avg = 0.2125 fifteen_min_avg = 0.23375 ] cpu_share = 0.464583 ] ] 2025-09-01 Mon 02:10:54:418.976 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011139814912, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: VrouterStats 2025-09-01 Mon 02:10:54:419.000 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011139814912, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:54:419.044 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011139814912, Pid 113882]: SANDESH: Send FAILED: 1756692654418785 [SYS_NOTICE]: ComputeCpuStateTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3993-1 cpu_info= [ [ [ mem_virt = 32598 cpu_share = 0.464583 used_sys_mem = 412901704 one_min_cpuload = 0.1375 mem_res = 0 ], ] ] ] 2025-09-01 Mon 02:10:54:419.063 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011139814912, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: ComputeCpuStateTrace 2025-09-01 Mon 02:10:54:419.085 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011139814912, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:54:419.460 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011139814912, Pid 113882]: SANDESH: Send FAILED: 1756692654418842 [SYS_NOTICE]: VrouterStats: data= [ name = cn-jenkins-deploy-platform-ansible-os-3993-1 in_tpkts = 168 in_bytes = 7224 out_tpkts = 172 out_bytes = 7504 raw_vhost_stats= [ name = vhost0 in_pkts = 168 in_bytes = 7224 out_pkts = 172 out_bytes = 7504 speed = -1 duplexity = -1 ] raw_vhost_drop_stats= [ ds_discard = 0 ds_pull = 0 ds_invalid_if = 0 ds_invalid_arp = 0 ds_trap_no_if = 0 ds_nowhere_to_go = 0 ds_flow_queue_limit_exceeded = 0 ds_flow_no_memory = 0 ds_flow_invalid_protocol = 0 ds_flow_nat_no_rflow = 0 ds_flow_action_drop = 0 ds_flow_action_invalid = 0 ds_flow_unusable = 0 ds_flow_table_full = 0 ds_interface_tx_discard = 0 ds_interface_drop = 0 ds_duplicated = 0 ds_push = 0 ds_ttl_exceeded = 0 ds_invalid_nh = 0 ds_invalid_label = 0 ds_invalid_protocol = 0 ds_interface_rx_discard = 0 ds_invalid_mcast_source = 0 ds_head_alloc_fail = 0 ds_pcow_fail = 0 ds_mcast_clone_fail = 0 ds_rewrite_fail = 0 ds_misc = 0 ds_invalid_packet = 0 ds_cksum_err = 0 ds_no_fmd = 0 ds_invalid_vnid = 0 ds_frag_err = 0 ds_invalid_source = 0 ds_mcast_df_bit = 0 ds_l2_no_route = 0 ds_vlan_fwd_tx = 0 ds_vlan_fwd_enq = 0 ds_drop_new_flow = 0 ds_flow_evict = 0 ds_trap_original = 0 ds_no_memory = 0 ds_fragment_queue_fail = 0 ds_no_frag_entry = 0 ds_icmp_error = 0 ds_clone_fail = 0 ds_drop_pkts = 0 ] raw_drop_stats= [ ds_discard = 0 ds_pull = 0 ds_invalid_if = 0 ds_invalid_arp = 0 ds_trap_no_if = 0 ds_nowhere_to_go = 0 ds_flow_queue_limit_exceeded = 0 ds_flow_no_memory = 0 ds_flow_invalid_protocol = 0 ds_flow_nat_no_rflow = 0 ds_flow_action_drop = 0 ds_flow_action_invalid = 0 ds_flow_unusable = 0 ds_flow_table_full = 0 ds_interface_tx_discard = 0 ds_interface_drop = 0 ds_duplicated = 0 ds_push = 0 ds_ttl_exceeded = 0 ds_invalid_nh = 0 ds_invalid_label = 0 ds_invalid_protocol = 0 ds_interface_rx_discard = 0 ds_invalid_mcast_source = 0 ds_head_alloc_fail = 0 ds_pcow_fail = 0 ds_mcast_clone_fail = 0 ds_rewrite_fail = 0 ds_misc = 0 ds_invalid_packet = 0 ds_cksum_err = 0 ds_no_fmd = 0 ds_invalid_vnid = 0 ds_frag_err = 0 ds_invalid_source = 0 ds_mcast_df_bit = 0 ds_l2_no_route = 0 ds_vlan_fwd_tx = 0 ds_vlan_fwd_enq = 0 ds_drop_new_flow = 0 ds_flow_evict = 0 ds_trap_original = 0 ds_no_memory = 0 ds_fragment_queue_fail = 0 ds_no_frag_entry = 0 ds_icmp_error = 0 ds_clone_fail = 0 ds_drop_pkts = 0 ] flow_rate= [ added_flows = 0 max_flow_adds_per_second = 0 min_flow_adds_per_second = 0 deleted_flows = 0 max_flow_deletes_per_second = 0 min_flow_deletes_per_second = 0 active_flows = 0 hold_flows = 0 ] raw_phy_if_stats= [ [ _iter262->first = ens6 [ in_pkts = 10 in_bytes = 700 out_pkts = 168 out_bytes = 7224 ], ] ] raw_phy_if_drop_stats= [ [ _iter263->first = ens6 [ ds_discard = 0 ds_pull = 0 ds_invalid_if = 0 ds_invalid_arp = 0 ds_trap_no_if = 0 ds_nowhere_to_go = 0 ds_flow_queue_limit_exceeded = 0 ds_flow_no_memory = 0 ds_flow_invalid_protocol = 0 ds_flow_nat_no_rflow = 0 ds_flow_action_drop = 0 ds_flow_action_invalid = 0 ds_flow_unusable = 0 ds_flow_table_full = 0 ds_interface_tx_discard = 0 ds_interface_drop = 0 ds_duplicated = 0 ds_push = 0 ds_ttl_exceeded = 0 ds_invalid_nh = 0 ds_invalid_label = 0 ds_invalid_protocol = 0 ds_interface_rx_discard = 0 ds_invalid_mcast_source = 0 ds_head_alloc_fail = 0 ds_pcow_fail = 0 ds_mcast_clone_fail = 0 ds_rewrite_fail = 0 ds_misc = 0 ds_invalid_packet = 0 ds_cksum_err = 0 ds_no_fmd = 0 ds_invalid_vnid = 0 ds_frag_err = 0 ds_invalid_source = 0 ds_mcast_df_bit = 0 ds_l2_no_route = 0 ds_vlan_fwd_tx = 0 ds_vlan_fwd_enq = 0 ds_drop_new_flow = 0 ds_flow_evict = 0 ds_trap_original = 0 ds_no_memory = 0 ds_fragment_queue_fail = 0 ds_no_frag_entry = 0 ds_icmp_error = 0 ds_clone_fail = 0 ds_drop_pkts = 0 ], ] ] ] 2025-09-01 Mon 02:10:54:419.509 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011139814912, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: VrouterStats 2025-09-01 Mon 02:10:54:419.539 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011139814912, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:54:419.611 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011139814912, Pid 113882]: SANDESH: Send FAILED: 1756692654418906 [SYS_NOTICE]: VrouterControlStatsTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3993-1 raw_xmpp_stats= [ [ _iter306->first = 10.20.0.14 [ reconnects = 0 in_msgs = 0 out_msgs = 0 ], ] ] raw_ifmap_stats= [ [ _iter307->first = link_delete_parse_errors _iter307->second = 0, _iter307->first = link_update_parse_errors _iter307->second = 0, _iter307->first = node_delete_parse_errors _iter307->second = 0, _iter307->first = node_update_parse_errors _iter307->second = 0, ] ] raw_rt_table_size= [ [ _iter308->first = default-domain:default-project:ip-fabric:__default__ [ inet4_unicast = 4 inet4_multicast = 0 evpn = 0 bridge = 2 inet6_unicast = 2 ], _iter308->first = default-domain:default-project:ip-fabric:ip-fabric [ inet4_unicast = 2 inet4_multicast = 0 evpn = 1 bridge = 2 inet6_unicast = 0 ], ] ] ] 2025-09-01 Mon 02:10:54:419.630 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011139814912, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: VrouterControlStatsTrace 2025-09-01 Mon 02:10:55:427.756 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011160806912, Pid 113882]: Processing scm::EvIdleHoldTimerExpired in state Idle 2025-09-01 Mon 02:10:55:428.056 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011160806912, Pid 113882]: Connect : Start Connect timer 10.0.0.56:8086 2025-09-01 Mon 02:10:55:428.105 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011171994240, Pid 113882]: -(-1) OnSessionEvent TCP Connect Failed 2025-09-01 Mon 02:10:55:428.155 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011171994240, Pid 113882]: -(-1):EvTcpConnectFail 2025-09-01 Mon 02:10:55:428.215 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011160806912, Pid 113882]: Processing scm::EvSandeshSend in state Connect 2025-09-01 Mon 02:10:55:428.249 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011160806912, Pid 113882]: SANDESH: Send FAILED: 1756692655427853 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-3993-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.14 server_addrs= [ [ (*_iter6) = 10.20.0.14:5269, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.14 server_addrs= [ [ (*_iter6) = 10.20.0.14:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.56:8086, ] ] status = Initializing description = Idle : EvIdleHoldTimerExpired -> Connect ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.14, XMPP:dns-server:10.20.0.14, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:10:55:428.259 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011160806912, Pid 113882]: Wrong state: Connect for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:10:55:428.270 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011160806912, Pid 113882]: Processing scm::EvTcpConnectFail in state Connect 2025-09-01 Mon 02:10:55:428.279 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011160806912, Pid 113882]: Connect : EvTcpConnectFail 2025-09-01 Mon 02:10:55:428.329 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011160806912, Pid 113882]: TCP [SYS_ERR]: TcpSessionMessageLog: Session -(-1) > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:10:55:428.364 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011160806912, Pid 113882]: Idle 2025-09-01 Mon 02:10:55:428.466 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011160806912, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:55:428.634 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011160806912, Pid 113882]: SANDESH: Send FAILED: 1756692655428167 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3993-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Connect successful_connections = 0 pid = 113882 http_port = 8085 start_time = 1756692234402667 collector_name = collector_ip = 10.0.0.56:8086 collector_list= [ [ (*_iter40) = 10.0.0.56:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ComputeCpuStateTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 7 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 41 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 66 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 102 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterControlStatsTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 14 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 21 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 148 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 38 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 143, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 302, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:10:55:428.659 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011160806912, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:10:55:428.670 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011160806912, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:55:428.690 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011160806912, Pid 113882]: SANDESH: Send FAILED: 1756692655428304 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-3993-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.14 server_addrs= [ [ (*_iter6) = 10.20.0.14:5269, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.14 server_addrs= [ [ (*_iter6) = 10.20.0.14:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.14, XMPP:dns-server:10.20.0.14, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:10:55:428.702 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011160806912, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:10:55:428.711 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011160806912, Pid 113882]: Processing scm::EvTcpDeleteSession in state Idle 2025-09-01 Mon 02:10:55:428.746 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011160806912, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:55:428.895 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011160806912, Pid 113882]: SANDESH: Send FAILED: 1756692655428411 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3993-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Idle successful_connections = 0 pid = 113882 http_port = 8085 start_time = 1756692234402667 collector_name = collector_ip = 10.0.0.56:8086 collector_list= [ [ (*_iter40) = 10.0.0.56:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ComputeCpuStateTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 7 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 42 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 66 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 103 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterControlStatsTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 14 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 21 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 148 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 38 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 144, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 303, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:10:55:428.913 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011160806912, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:11:04:083.839 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011171994240, Pid 113882]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.0.0.56:8085::10.0.0.56:34038 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 255 2025-09-01 Mon 02:11:15:292.504 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011148211712, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:15:292.627 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011148211712, Pid 113882]: SANDESH: Send FAILED: 1756692675292383 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.14 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:15:292.655 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011148211712, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:15:292.911 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011171994240, Pid 113882]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.14 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:11:15:292.913 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011139814912, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:15:293.104 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011160806912, Pid 113882]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:11:15:293.119 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011139814912, Pid 113882]: SANDESH: Send FAILED: 1756692675292836 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.14 state = Connect event = Connect Event local_port = 43877 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:15:293.259 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011139814912, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:15:293.312 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011139814912, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:15:293.346 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011139814912, Pid 113882]: SANDESH: Send FAILED: 1756692675293244 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.14 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:15:293.365 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011139814912, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:15:293.385 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011139814912, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:15:293.411 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011139814912, Pid 113882]: SANDESH: Send FAILED: 1756692675293260 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.14 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:15:293.428 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011139814912, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:17:002.894 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011171994240, Pid 113882]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.0.0.56:8085::10.0.0.56:59274 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 255 2025-09-01 Mon 02:11:17:856.404 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011152410112, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:17:856.574 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011152410112, Pid 113882]: SANDESH: Send FAILED: 1756692677856346 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.14 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:17:856.601 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011152410112, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:17:856.765 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011135616512, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:17:856.778 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011171994240, Pid 113882]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.14 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:11:17:856.932 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011135616512, Pid 113882]: SANDESH: Send FAILED: 1756692677856707 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.14 state = Connect event = Connect Event local_port = 44873 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:17:856.961 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011148211712, Pid 113882]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:11:17:856.994 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011135616512, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:17:857.222 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011152410112, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:17:857.259 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011152410112, Pid 113882]: SANDESH: Send FAILED: 1756692677857126 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.14 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:17:857.296 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011152410112, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:17:857.317 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011152410112, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:17:857.342 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011152410112, Pid 113882]: SANDESH: Send FAILED: 1756692677857213 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.14 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:17:857.357 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011152410112, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:24:412.503 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011152410112, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:24:412.597 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011152410112, Pid 113882]: SANDESH: Send FAILED: 1756692684412071 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3993-1:Compute:contrail-vrouter-agent:0 sm_queue_count = 0 sm_stats= [ ev_stats= [ [ [ event = scm::EvCollectorUpdate enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvIdleHoldTimerExpired enqueues = 18 dequeues = 18 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvSandeshSend enqueues = 314 dequeues = 314 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvStart enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpConnectFail enqueues = 19 dequeues = 19 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpDeleteSession enqueues = 19 dequeues = 19 enqueue_fails = 0 dequeue_fails = 0 ], [ event = enqueues = 372 dequeues = 372 enqueue_fails = 0 dequeue_fails = 0 ], ] ] state = Idle state_since = 1756692655428363 last_state = Idle last_event = scm::EvSandeshSend last_event_at = 1756692677857314 ] max_sm_queue_count = 5 ] 2025-09-01 Mon 02:11:24:412.610 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011152410112, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:11:24:412.619 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011152410112, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:24:412.740 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011152410112, Pid 113882]: SANDESH: Send FAILED: 1756692684412325 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3993-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Idle successful_connections = 0 pid = 113882 http_port = 8085 start_time = 1756692234402667 collector_name = collector_ip = 10.0.0.56:8086 collector_list= [ [ (*_iter40) = 10.0.0.56:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ComputeCpuStateTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 7 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 43 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 68 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 107 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterControlStatsTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 14 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 21 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 156 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 40 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 150, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 314, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:11:24:412.765 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011152410112, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:11:24:419.441 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011139814912, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:24:419.866 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011139814912, Pid 113882]: SANDESH: Send FAILED: 1756692684419297 [SYS_NOTICE]: VrouterStats: data= [ name = cn-jenkins-deploy-platform-ansible-os-3993-1 in_tpkts = 171 in_bytes = 7350 out_tpkts = 175 out_bytes = 7630 raw_vhost_stats= [ name = vhost0 in_pkts = 171 in_bytes = 7350 out_pkts = 175 out_bytes = 7630 speed = -1 duplexity = -1 ] raw_vhost_drop_stats= [ ds_discard = 0 ds_pull = 0 ds_invalid_if = 0 ds_invalid_arp = 0 ds_trap_no_if = 0 ds_nowhere_to_go = 0 ds_flow_queue_limit_exceeded = 0 ds_flow_no_memory = 0 ds_flow_invalid_protocol = 0 ds_flow_nat_no_rflow = 0 ds_flow_action_drop = 0 ds_flow_action_invalid = 0 ds_flow_unusable = 0 ds_flow_table_full = 0 ds_interface_tx_discard = 0 ds_interface_drop = 0 ds_duplicated = 0 ds_push = 0 ds_ttl_exceeded = 0 ds_invalid_nh = 0 ds_invalid_label = 0 ds_invalid_protocol = 0 ds_interface_rx_discard = 0 ds_invalid_mcast_source = 0 ds_head_alloc_fail = 0 ds_pcow_fail = 0 ds_mcast_clone_fail = 0 ds_rewrite_fail = 0 ds_misc = 0 ds_invalid_packet = 0 ds_cksum_err = 0 ds_no_fmd = 0 ds_invalid_vnid = 0 ds_frag_err = 0 ds_invalid_source = 0 ds_mcast_df_bit = 0 ds_l2_no_route = 0 ds_vlan_fwd_tx = 0 ds_vlan_fwd_enq = 0 ds_drop_new_flow = 0 ds_flow_evict = 0 ds_trap_original = 0 ds_no_memory = 0 ds_fragment_queue_fail = 0 ds_no_frag_entry = 0 ds_icmp_error = 0 ds_clone_fail = 0 ds_drop_pkts = 0 ] raw_drop_stats= [ ds_discard = 0 ds_pull = 0 ds_invalid_if = 0 ds_invalid_arp = 0 ds_trap_no_if = 0 ds_nowhere_to_go = 0 ds_flow_queue_limit_exceeded = 0 ds_flow_no_memory = 0 ds_flow_invalid_protocol = 0 ds_flow_nat_no_rflow = 0 ds_flow_action_drop = 0 ds_flow_action_invalid = 0 ds_flow_unusable = 0 ds_flow_table_full = 0 ds_interface_tx_discard = 0 ds_interface_drop = 0 ds_duplicated = 0 ds_push = 0 ds_ttl_exceeded = 0 ds_invalid_nh = 0 ds_invalid_label = 0 ds_invalid_protocol = 0 ds_interface_rx_discard = 0 ds_invalid_mcast_source = 0 ds_head_alloc_fail = 0 ds_pcow_fail = 0 ds_mcast_clone_fail = 0 ds_rewrite_fail = 0 ds_misc = 0 ds_invalid_packet = 0 ds_cksum_err = 0 ds_no_fmd = 0 ds_invalid_vnid = 0 ds_frag_err = 0 ds_invalid_source = 0 ds_mcast_df_bit = 0 ds_l2_no_route = 0 ds_vlan_fwd_tx = 0 ds_vlan_fwd_enq = 0 ds_drop_new_flow = 0 ds_flow_evict = 0 ds_trap_original = 0 ds_no_memory = 0 ds_fragment_queue_fail = 0 ds_no_frag_entry = 0 ds_icmp_error = 0 ds_clone_fail = 0 ds_drop_pkts = 0 ] total_in_bandwidth_utilization = 9.75782e-17 flow_rate= [ added_flows = 0 max_flow_adds_per_second = 0 min_flow_adds_per_second = 0 deleted_flows = 0 max_flow_deletes_per_second = 0 min_flow_deletes_per_second = 0 active_flows = 0 hold_flows = 0 ] phy_band_in_bps= [ [ _iter260->first = ens6 _iter260->second = 18, ] ] phy_band_out_bps= [ [ _iter261->first = ens6 _iter261->second = 33, ] ] raw_phy_if_stats= [ [ _iter262->first = ens6 [ in_pkts = 10 in_bytes = 700 out_pkts = 171 out_bytes = 7350 ], ] ] raw_phy_if_drop_stats= [ [ _iter263->first = ens6 [ ds_discard = 0 ds_pull = 0 ds_invalid_if = 0 ds_invalid_arp = 0 ds_trap_no_if = 0 ds_nowhere_to_go = 0 ds_flow_queue_limit_exceeded = 0 ds_flow_no_memory = 0 ds_flow_invalid_protocol = 0 ds_flow_nat_no_rflow = 0 ds_flow_action_drop = 0 ds_flow_action_invalid = 0 ds_flow_unusable = 0 ds_flow_table_full = 0 ds_interface_tx_discard = 0 ds_interface_drop = 0 ds_duplicated = 0 ds_push = 0 ds_ttl_exceeded = 0 ds_invalid_nh = 0 ds_invalid_label = 0 ds_invalid_protocol = 0 ds_interface_rx_discard = 0 ds_invalid_mcast_source = 0 ds_head_alloc_fail = 0 ds_pcow_fail = 0 ds_mcast_clone_fail = 0 ds_rewrite_fail = 0 ds_misc = 0 ds_invalid_packet = 0 ds_cksum_err = 0 ds_no_fmd = 0 ds_invalid_vnid = 0 ds_frag_err = 0 ds_invalid_source = 0 ds_mcast_df_bit = 0 ds_l2_no_route = 0 ds_vlan_fwd_tx = 0 ds_vlan_fwd_enq = 0 ds_drop_new_flow = 0 ds_flow_evict = 0 ds_trap_original = 0 ds_no_memory = 0 ds_fragment_queue_fail = 0 ds_no_frag_entry = 0 ds_icmp_error = 0 ds_clone_fail = 0 ds_drop_pkts = 0 ], ] ] ] 2025-09-01 Mon 02:11:24:419.935 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011139814912, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: VrouterStats 2025-09-01 Mon 02:11:24:419.975 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011139814912, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:24:420.053 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011139814912, Pid 113882]: SANDESH: Send FAILED: 1756692684419434 [SYS_NOTICE]: VrouterControlStatsTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3993-1 raw_xmpp_stats= [ [ _iter306->first = 10.20.0.14 [ reconnects = 0 in_msgs = 0 out_msgs = 0 ], ] ] raw_ifmap_stats= [ [ _iter307->first = link_delete_parse_errors _iter307->second = 0, _iter307->first = link_update_parse_errors _iter307->second = 0, _iter307->first = node_delete_parse_errors _iter307->second = 0, _iter307->first = node_update_parse_errors _iter307->second = 0, ] ] raw_rt_table_size= [ [ _iter308->first = default-domain:default-project:ip-fabric:__default__ [ inet4_unicast = 4 inet4_multicast = 0 evpn = 0 bridge = 2 inet6_unicast = 2 ], _iter308->first = default-domain:default-project:ip-fabric:ip-fabric [ inet4_unicast = 2 inet4_multicast = 0 evpn = 1 bridge = 2 inet6_unicast = 0 ], ] ] ] 2025-09-01 Mon 02:11:24:420.075 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011139814912, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: VrouterControlStatsTrace 2025-09-01 Mon 02:11:25:428.643 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Processing scm::EvIdleHoldTimerExpired in state Idle 2025-09-01 Mon 02:11:25:429.169 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Connect : Start Connect timer 10.0.0.56:8086 2025-09-01 Mon 02:11:25:429.240 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011171994240, Pid 113882]: -(-1) OnSessionEvent TCP Connect Failed 2025-09-01 Mon 02:11:25:429.294 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011171994240, Pid 113882]: -(-1):EvTcpConnectFail 2025-09-01 Mon 02:11:25:429.490 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Processing scm::EvSandeshSend in state Connect 2025-09-01 Mon 02:11:25:429.564 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: SANDESH: Send FAILED: 1756692685428813 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-3993-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.14 server_addrs= [ [ (*_iter6) = 10.20.0.14:5269, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.14 server_addrs= [ [ (*_iter6) = 10.20.0.14:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.56:8086, ] ] status = Initializing description = Idle : EvIdleHoldTimerExpired -> Connect ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.14, XMPP:dns-server:10.20.0.14, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:11:25:429.598 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Wrong state: Connect for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:11:25:429.636 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Processing scm::EvTcpConnectFail in state Connect 2025-09-01 Mon 02:11:25:429.665 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Connect : EvTcpConnectFail 2025-09-01 Mon 02:11:25:429.811 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: TCP [SYS_ERR]: TcpSessionMessageLog: Session -(-1) > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:11:25:429.896 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Idle 2025-09-01 Mon 02:11:25:430.038 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:25:430.501 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: SANDESH: Send FAILED: 1756692685429448 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3993-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Connect successful_connections = 0 pid = 113882 http_port = 8085 start_time = 1756692234402667 collector_name = collector_ip = 10.0.0.56:8086 collector_list= [ [ (*_iter40) = 10.0.0.56:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ComputeCpuStateTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 7 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 43 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 70 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 107 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterControlStatsTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 15 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 22 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 156 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 40 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 150, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 318, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:11:25:430.593 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:11:25:430.626 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:25:430.691 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: SANDESH: Send FAILED: 1756692685429743 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-3993-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.14 server_addrs= [ [ (*_iter6) = 10.20.0.14:5269, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.14 server_addrs= [ [ (*_iter6) = 10.20.0.14:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.14, XMPP:dns-server:10.20.0.14, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:11:25:430.719 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:11:25:430.754 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Processing scm::EvTcpDeleteSession in state Idle 2025-09-01 Mon 02:11:25:430.836 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:25:431.244 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: SANDESH: Send FAILED: 1756692685430012 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3993-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Idle successful_connections = 0 pid = 113882 http_port = 8085 start_time = 1756692234402667 collector_name = collector_ip = 10.0.0.56:8086 collector_list= [ [ (*_iter40) = 10.0.0.56:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ComputeCpuStateTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 7 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 44 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 70 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 108 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterControlStatsTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 15 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 22 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 156 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 40 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 151, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 319, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:11:25:431.296 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:11:29:835.989 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011171994240, Pid 113882]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.0.0.56:8085::10.0.0.56:56344 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 255 2025-09-01 Mon 02:11:37:439.048 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011171994240, Pid 113882]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.20.0.14:8085::10.20.0.14:60358 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 255 2025-09-01 Mon 02:11:37:457.219 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011171994240, Pid 113882]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.20.0.14:8085::10.20.0.14:60370 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 255 2025-09-01 Mon 02:11:43:055.885 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011171994240, Pid 113882]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.0.0.56:8085::10.0.0.56:43638 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 255 2025-09-01 Mon 02:11:46:613.676 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011152410112, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:46:613.821 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011152410112, Pid 113882]: SANDESH: Send FAILED: 1756692706613605 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.14 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:46:613.864 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011152410112, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:46:614.015 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011148211712, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:46:614.029 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011171994240, Pid 113882]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.14 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:11:46:614.126 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011148211712, Pid 113882]: SANDESH: Send FAILED: 1756692706613943 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.14 state = Connect event = Connect Event local_port = 38735 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:46:614.250 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011148211712, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:46:614.308 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011152410112, Pid 113882]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:11:46:614.502 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011148211712, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:46:614.539 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011148211712, Pid 113882]: SANDESH: Send FAILED: 1756692706614413 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.14 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:46:614.558 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011148211712, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:46:614.580 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011148211712, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:46:614.606 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011148211712, Pid 113882]: SANDESH: Send FAILED: 1756692706614470 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.14 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:46:614.623 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011148211712, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:48:637.576 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011152410112, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:48:637.692 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011152410112, Pid 113882]: SANDESH: Send FAILED: 1756692708637512 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.14 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:48:637.720 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011152410112, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:48:637.994 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011171994240, Pid 113882]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.14 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:11:48:638.010 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011139814912, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:48:638.232 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:11:48:638.236 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011139814912, Pid 113882]: SANDESH: Send FAILED: 1756692708637929 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.14 state = Connect event = Connect Event local_port = 37857 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:48:638.393 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011139814912, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:48:638.434 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011139814912, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:48:638.486 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011139814912, Pid 113882]: SANDESH: Send FAILED: 1756692708638386 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.14 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:48:638.514 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011139814912, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:48:638.547 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011139814912, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:48:638.592 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011139814912, Pid 113882]: SANDESH: Send FAILED: 1756692708638409 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.14 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:48:638.619 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011139814912, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:54:412.869 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011139814912, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:54:421.487 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011139814912, Pid 113882]: SANDESH: Send FAILED: 1756692714412587 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3993-1:Compute:contrail-vrouter-agent:0 sm_queue_count = 0 sm_stats= [ ev_stats= [ [ [ event = scm::EvCollectorUpdate enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvIdleHoldTimerExpired enqueues = 19 dequeues = 19 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvSandeshSend enqueues = 330 dequeues = 330 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvStart enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpConnectFail enqueues = 20 dequeues = 20 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpDeleteSession enqueues = 20 dequeues = 20 enqueue_fails = 0 dequeue_fails = 0 ], [ event = enqueues = 391 dequeues = 391 enqueue_fails = 0 dequeue_fails = 0 ], ] ] state = Idle state_since = 1756692685429893 last_state = Idle last_event = scm::EvSandeshSend last_event_at = 1756692708638541 ] max_sm_queue_count = 5 ] 2025-09-01 Mon 02:11:54:421.570 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011139814912, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:11:54:421.602 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011139814912, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:54:421.992 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011139814912, Pid 113882]: SANDESH: Send FAILED: 1756692714412770 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3993-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Idle successful_connections = 0 pid = 113882 http_port = 8085 start_time = 1756692234402667 collector_name = collector_ip = 10.0.0.56:8086 collector_list= [ [ (*_iter40) = 10.0.0.56:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ComputeCpuStateTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 7 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 45 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 72 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 114 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterControlStatsTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 15 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 22 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 164 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 42 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 159, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 330, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:11:54:422.038 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011139814912, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:11:54:422.066 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011139814912, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:54:422.140 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011139814912, Pid 113882]: SANDESH: Send FAILED: 1756692714419693 [SYS_NOTICE]: VrouterStats: data= [ name = cn-jenkins-deploy-platform-ansible-os-3993-1 cpu_info= [ num_cpu = 8 sys_mem_info= [ total = 32 used = 0 free = 0 buffers = 0 cached = 3882065592 node_type = ] meminfo= [ virt = 32598 peakvirt = 0 res = 0 ] cpuload= [ one_min_avg = 0.1325 five_min_avg = 0.19625 fifteen_min_avg = 0.22625 ] cpu_share = 0.472917 ] ] 2025-09-01 Mon 02:11:54:422.162 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011139814912, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: VrouterStats 2025-09-01 Mon 02:11:54:422.212 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011139814912, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:54:422.257 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011139814912, Pid 113882]: SANDESH: Send FAILED: 1756692714419712 [SYS_NOTICE]: ComputeCpuStateTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3993-1 cpu_info= [ [ [ mem_virt = 32598 cpu_share = 0.472917 used_sys_mem = 412901704 one_min_cpuload = 0.1325 mem_res = 0 ], ] ] ] 2025-09-01 Mon 02:11:54:422.276 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011139814912, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: ComputeCpuStateTrace 2025-09-01 Mon 02:11:54:422.298 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011139814912, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:54:422.613 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011139814912, Pid 113882]: SANDESH: Send FAILED: 1756692714419756 [SYS_NOTICE]: VrouterStats: data= [ name = cn-jenkins-deploy-platform-ansible-os-3993-1 in_tpkts = 174 in_bytes = 7476 out_tpkts = 178 out_bytes = 7756 raw_vhost_stats= [ name = vhost0 in_pkts = 174 in_bytes = 7476 out_pkts = 178 out_bytes = 7756 speed = -1 duplexity = -1 ] raw_vhost_drop_stats= [ ds_discard = 0 ds_pull = 0 ds_invalid_if = 0 ds_invalid_arp = 0 ds_trap_no_if = 0 ds_nowhere_to_go = 0 ds_flow_queue_limit_exceeded = 0 ds_flow_no_memory = 0 ds_flow_invalid_protocol = 0 ds_flow_nat_no_rflow = 0 ds_flow_action_drop = 0 ds_flow_action_invalid = 0 ds_flow_unusable = 0 ds_flow_table_full = 0 ds_interface_tx_discard = 0 ds_interface_drop = 0 ds_duplicated = 0 ds_push = 0 ds_ttl_exceeded = 0 ds_invalid_nh = 0 ds_invalid_label = 0 ds_invalid_protocol = 0 ds_interface_rx_discard = 0 ds_invalid_mcast_source = 0 ds_head_alloc_fail = 0 ds_pcow_fail = 0 ds_mcast_clone_fail = 0 ds_rewrite_fail = 0 ds_misc = 0 ds_invalid_packet = 0 ds_cksum_err = 0 ds_no_fmd = 0 ds_invalid_vnid = 0 ds_frag_err = 0 ds_invalid_source = 0 ds_mcast_df_bit = 0 ds_l2_no_route = 0 ds_vlan_fwd_tx = 0 ds_vlan_fwd_enq = 0 ds_drop_new_flow = 0 ds_flow_evict = 0 ds_trap_original = 0 ds_no_memory = 0 ds_fragment_queue_fail = 0 ds_no_frag_entry = 0 ds_icmp_error = 0 ds_clone_fail = 0 ds_drop_pkts = 0 ] raw_drop_stats= [ ds_discard = 0 ds_pull = 0 ds_invalid_if = 0 ds_invalid_arp = 0 ds_trap_no_if = 0 ds_nowhere_to_go = 0 ds_flow_queue_limit_exceeded = 0 ds_flow_no_memory = 0 ds_flow_invalid_protocol = 0 ds_flow_nat_no_rflow = 0 ds_flow_action_drop = 0 ds_flow_action_invalid = 0 ds_flow_unusable = 0 ds_flow_table_full = 0 ds_interface_tx_discard = 0 ds_interface_drop = 0 ds_duplicated = 0 ds_push = 0 ds_ttl_exceeded = 0 ds_invalid_nh = 0 ds_invalid_label = 0 ds_invalid_protocol = 0 ds_interface_rx_discard = 0 ds_invalid_mcast_source = 0 ds_head_alloc_fail = 0 ds_pcow_fail = 0 ds_mcast_clone_fail = 0 ds_rewrite_fail = 0 ds_misc = 0 ds_invalid_packet = 0 ds_cksum_err = 0 ds_no_fmd = 0 ds_invalid_vnid = 0 ds_frag_err = 0 ds_invalid_source = 0 ds_mcast_df_bit = 0 ds_l2_no_route = 0 ds_vlan_fwd_tx = 0 ds_vlan_fwd_enq = 0 ds_drop_new_flow = 0 ds_flow_evict = 0 ds_trap_original = 0 ds_no_memory = 0 ds_fragment_queue_fail = 0 ds_no_frag_entry = 0 ds_icmp_error = 0 ds_clone_fail = 0 ds_drop_pkts = 0 ] flow_rate= [ added_flows = 0 max_flow_adds_per_second = 0 min_flow_adds_per_second = 0 deleted_flows = 0 max_flow_deletes_per_second = 0 min_flow_deletes_per_second = 0 active_flows = 0 hold_flows = 0 ] raw_phy_if_stats= [ [ _iter262->first = ens6 [ in_pkts = 10 in_bytes = 700 out_pkts = 174 out_bytes = 7476 ], ] ] raw_phy_if_drop_stats= [ [ _iter263->first = ens6 [ ds_discard = 0 ds_pull = 0 ds_invalid_if = 0 ds_invalid_arp = 0 ds_trap_no_if = 0 ds_nowhere_to_go = 0 ds_flow_queue_limit_exceeded = 0 ds_flow_no_memory = 0 ds_flow_invalid_protocol = 0 ds_flow_nat_no_rflow = 0 ds_flow_action_drop = 0 ds_flow_action_invalid = 0 ds_flow_unusable = 0 ds_flow_table_full = 0 ds_interface_tx_discard = 0 ds_interface_drop = 0 ds_duplicated = 0 ds_push = 0 ds_ttl_exceeded = 0 ds_invalid_nh = 0 ds_invalid_label = 0 ds_invalid_protocol = 0 ds_interface_rx_discard = 0 ds_invalid_mcast_source = 0 ds_head_alloc_fail = 0 ds_pcow_fail = 0 ds_mcast_clone_fail = 0 ds_rewrite_fail = 0 ds_misc = 0 ds_invalid_packet = 0 ds_cksum_err = 0 ds_no_fmd = 0 ds_invalid_vnid = 0 ds_frag_err = 0 ds_invalid_source = 0 ds_mcast_df_bit = 0 ds_l2_no_route = 0 ds_vlan_fwd_tx = 0 ds_vlan_fwd_enq = 0 ds_drop_new_flow = 0 ds_flow_evict = 0 ds_trap_original = 0 ds_no_memory = 0 ds_fragment_queue_fail = 0 ds_no_frag_entry = 0 ds_icmp_error = 0 ds_clone_fail = 0 ds_drop_pkts = 0 ], ] ] ] 2025-09-01 Mon 02:11:54:422.651 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011139814912, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: VrouterStats 2025-09-01 Mon 02:11:54:422.673 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011139814912, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:54:422.744 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011139814912, Pid 113882]: SANDESH: Send FAILED: 1756692714419803 [SYS_NOTICE]: VrouterControlStatsTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3993-1 raw_xmpp_stats= [ [ _iter306->first = 10.20.0.14 [ reconnects = 0 in_msgs = 0 out_msgs = 0 ], ] ] raw_ifmap_stats= [ [ _iter307->first = link_delete_parse_errors _iter307->second = 0, _iter307->first = link_update_parse_errors _iter307->second = 0, _iter307->first = node_delete_parse_errors _iter307->second = 0, _iter307->first = node_update_parse_errors _iter307->second = 0, ] ] raw_rt_table_size= [ [ _iter308->first = default-domain:default-project:ip-fabric:__default__ [ inet4_unicast = 4 inet4_multicast = 0 evpn = 0 bridge = 2 inet6_unicast = 2 ], _iter308->first = default-domain:default-project:ip-fabric:ip-fabric [ inet4_unicast = 2 inet4_multicast = 0 evpn = 1 bridge = 2 inet6_unicast = 0 ], ] ] ] 2025-09-01 Mon 02:11:54:422.771 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011139814912, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: VrouterControlStatsTrace 2025-09-01 Mon 02:11:55:430.052 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011160806912, Pid 113882]: Processing scm::EvIdleHoldTimerExpired in state Idle 2025-09-01 Mon 02:11:55:430.316 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011160806912, Pid 113882]: Connect : Start Connect timer 10.0.0.56:8086 2025-09-01 Mon 02:11:55:430.393 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011171994240, Pid 113882]: -(-1) OnSessionEvent TCP Connect Failed 2025-09-01 Mon 02:11:55:430.436 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011160806912, Pid 113882]: Processing scm::EvSandeshSend in state Connect 2025-09-01 Mon 02:11:55:430.489 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011171994240, Pid 113882]: -(-1):EvTcpConnectFail 2025-09-01 Mon 02:11:55:430.525 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011160806912, Pid 113882]: SANDESH: Send FAILED: 1756692715430140 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-3993-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.14 server_addrs= [ [ (*_iter6) = 10.20.0.14:5269, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.14 server_addrs= [ [ (*_iter6) = 10.20.0.14:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.56:8086, ] ] status = Initializing description = Idle : EvIdleHoldTimerExpired -> Connect ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.14, XMPP:dns-server:10.20.0.14, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:11:55:430.569 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011160806912, Pid 113882]: Wrong state: Connect for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:11:55:430.579 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011160806912, Pid 113882]: Processing scm::EvSandeshSend in state Connect 2025-09-01 Mon 02:11:55:430.719 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011160806912, Pid 113882]: SANDESH: Send FAILED: 1756692715430413 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3993-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Connect successful_connections = 0 pid = 113882 http_port = 8085 start_time = 1756692234402667 collector_name = collector_ip = 10.0.0.56:8086 collector_list= [ [ (*_iter40) = 10.0.0.56:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ComputeCpuStateTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 8 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 45 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 74 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 114 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterControlStatsTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 16 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 24 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 164 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 42 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 159, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 336, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:11:55:430.741 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011160806912, Pid 113882]: Wrong state: Connect for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:11:55:430.753 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011160806912, Pid 113882]: Processing scm::EvTcpConnectFail in state Connect 2025-09-01 Mon 02:11:55:430.760 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011160806912, Pid 113882]: Connect : EvTcpConnectFail 2025-09-01 Mon 02:11:55:430.804 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011160806912, Pid 113882]: TCP [SYS_ERR]: TcpSessionMessageLog: Session -(-1) > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:11:55:430.833 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011160806912, Pid 113882]: Idle 2025-09-01 Mon 02:11:55:430.890 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011160806912, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:55:430.909 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011160806912, Pid 113882]: SANDESH: Send FAILED: 1756692715430782 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-3993-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.14 server_addrs= [ [ (*_iter6) = 10.20.0.14:5269, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.14 server_addrs= [ [ (*_iter6) = 10.20.0.14:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.14, XMPP:dns-server:10.20.0.14, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:11:55:430.916 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011160806912, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:11:55:430.923 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011160806912, Pid 113882]: Processing scm::EvTcpDeleteSession in state Idle 2025-09-01 Mon 02:11:55:430.948 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011160806912, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:55:431.090 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011160806912, Pid 113882]: SANDESH: Send FAILED: 1756692715430876 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3993-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Idle successful_connections = 0 pid = 113882 http_port = 8085 start_time = 1756692234402667 collector_name = collector_ip = 10.0.0.56:8086 collector_list= [ [ (*_iter40) = 10.0.0.56:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ComputeCpuStateTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 8 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 46 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 75 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 115 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterControlStatsTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 16 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 24 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 164 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 42 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 160, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 338, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:11:55:431.342 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011160806912, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:11:56:081.965 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011171994240, Pid 113882]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.0.0.56:8085::10.0.0.56:40994 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 255 2025-09-01 Mon 02:12:09:338.094 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011171994240, Pid 113882]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.0.0.56:8085::10.0.0.56:38854 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 255 2025-09-01 Mon 02:12:14:694.812 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011160806912, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:14:694.926 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011160806912, Pid 113882]: SANDESH: Send FAILED: 1756692734694729 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.14 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:14:694.982 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011160806912, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:14:695.133 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011135616512, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:14:695.167 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011171994240, Pid 113882]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.14 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:12:14:695.277 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011135616512, Pid 113882]: SANDESH: Send FAILED: 1756692734695065 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.14 state = Connect event = Connect Event local_port = 46343 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:14:695.395 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011135616512, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:14:695.414 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011148211712, Pid 113882]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:12:14:695.593 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:14:695.649 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: SANDESH: Send FAILED: 1756692734695498 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.14 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:14:695.676 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:14:695.701 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:14:695.729 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: SANDESH: Send FAILED: 1756692734695550 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.14 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:14:695.747 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:19:958.781 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:19:958.855 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: SANDESH: Send FAILED: 1756692739958738 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.14 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:19:958.870 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:19:959.002 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011156608512, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:19:959.063 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011171994240, Pid 113882]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.14 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:12:19:959.067 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011156608512, Pid 113882]: SANDESH: Send FAILED: 1756692739958960 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.14 state = Connect event = Connect Event local_port = 45223 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:19:959.186 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011156608512, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:19:959.268 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011156608512, Pid 113882]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:12:19:959.313 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:19:959.332 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: SANDESH: Send FAILED: 1756692739959300 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.14 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:19:959.341 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:19:959.351 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:19:959.363 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: SANDESH: Send FAILED: 1756692739959305 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.14 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:19:959.371 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:22:412.002 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011171994240, Pid 113882]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.0.0.56:8085::10.0.0.56:42858 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 255 2025-09-01 Mon 02:12:24:413.366 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:24:413.592 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: SANDESH: Send FAILED: 1756692744413040 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3993-1:Compute:contrail-vrouter-agent:0 sm_queue_count = 0 sm_stats= [ ev_stats= [ [ [ event = scm::EvCollectorUpdate enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvIdleHoldTimerExpired enqueues = 20 dequeues = 20 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvSandeshSend enqueues = 348 dequeues = 348 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvStart enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpConnectFail enqueues = 21 dequeues = 21 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpDeleteSession enqueues = 21 dequeues = 21 enqueue_fails = 0 dequeue_fails = 0 ], [ event = enqueues = 412 dequeues = 412 enqueue_fails = 0 dequeue_fails = 0 ], ] ] state = Idle state_since = 1756692715430832 last_state = Idle last_event = scm::EvSandeshSend last_event_at = 1756692739959349 ] max_sm_queue_count = 5 ] 2025-09-01 Mon 02:12:24:413.657 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:12:24:413.685 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:24:414.218 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: SANDESH: Send FAILED: 1756692744413270 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3993-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Idle successful_connections = 0 pid = 113882 http_port = 8085 start_time = 1756692234402667 collector_name = collector_ip = 10.0.0.56:8086 collector_list= [ [ (*_iter40) = 10.0.0.56:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ComputeCpuStateTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 8 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 47 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 76 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 120 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterControlStatsTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 16 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 24 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 172 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 44 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 167, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 348, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:12:24:414.292 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:12:24:420.305 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011152410112, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:24:420.890 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011152410112, Pid 113882]: SANDESH: Send FAILED: 1756692744420226 [SYS_NOTICE]: VrouterStats: data= [ name = cn-jenkins-deploy-platform-ansible-os-3993-1 in_tpkts = 178 in_bytes = 7672 out_tpkts = 181 out_bytes = 7882 raw_vhost_stats= [ name = vhost0 in_pkts = 178 in_bytes = 7672 out_pkts = 181 out_bytes = 7882 speed = -1 duplexity = -1 ] raw_vhost_drop_stats= [ ds_discard = 0 ds_pull = 0 ds_invalid_if = 0 ds_invalid_arp = 0 ds_trap_no_if = 0 ds_nowhere_to_go = 0 ds_flow_queue_limit_exceeded = 0 ds_flow_no_memory = 0 ds_flow_invalid_protocol = 0 ds_flow_nat_no_rflow = 0 ds_flow_action_drop = 0 ds_flow_action_invalid = 0 ds_flow_unusable = 0 ds_flow_table_full = 0 ds_interface_tx_discard = 0 ds_interface_drop = 0 ds_duplicated = 0 ds_push = 0 ds_ttl_exceeded = 0 ds_invalid_nh = 0 ds_invalid_label = 0 ds_invalid_protocol = 0 ds_interface_rx_discard = 0 ds_invalid_mcast_source = 0 ds_head_alloc_fail = 0 ds_pcow_fail = 0 ds_mcast_clone_fail = 0 ds_rewrite_fail = 0 ds_misc = 0 ds_invalid_packet = 0 ds_cksum_err = 0 ds_no_fmd = 0 ds_invalid_vnid = 0 ds_frag_err = 0 ds_invalid_source = 0 ds_mcast_df_bit = 0 ds_l2_no_route = 0 ds_vlan_fwd_tx = 0 ds_vlan_fwd_enq = 0 ds_drop_new_flow = 0 ds_flow_evict = 0 ds_trap_original = 0 ds_no_memory = 0 ds_fragment_queue_fail = 0 ds_no_frag_entry = 0 ds_icmp_error = 0 ds_clone_fail = 0 ds_drop_pkts = 0 ] raw_drop_stats= [ ds_discard = 0 ds_pull = 0 ds_invalid_if = 0 ds_invalid_arp = 0 ds_trap_no_if = 0 ds_nowhere_to_go = 0 ds_flow_queue_limit_exceeded = 0 ds_flow_no_memory = 0 ds_flow_invalid_protocol = 0 ds_flow_nat_no_rflow = 0 ds_flow_action_drop = 0 ds_flow_action_invalid = 0 ds_flow_unusable = 0 ds_flow_table_full = 0 ds_interface_tx_discard = 0 ds_interface_drop = 0 ds_duplicated = 0 ds_push = 0 ds_ttl_exceeded = 0 ds_invalid_nh = 0 ds_invalid_label = 0 ds_invalid_protocol = 0 ds_interface_rx_discard = 0 ds_invalid_mcast_source = 0 ds_head_alloc_fail = 0 ds_pcow_fail = 0 ds_mcast_clone_fail = 0 ds_rewrite_fail = 0 ds_misc = 0 ds_invalid_packet = 0 ds_cksum_err = 0 ds_no_fmd = 0 ds_invalid_vnid = 0 ds_frag_err = 0 ds_invalid_source = 0 ds_mcast_df_bit = 0 ds_l2_no_route = 0 ds_vlan_fwd_tx = 0 ds_vlan_fwd_enq = 0 ds_drop_new_flow = 0 ds_flow_evict = 0 ds_trap_original = 0 ds_no_memory = 0 ds_fragment_queue_fail = 0 ds_no_frag_entry = 0 ds_icmp_error = 0 ds_clone_fail = 0 ds_drop_pkts = 0 ] total_in_bandwidth_utilization = 0 total_out_bandwidth_utilization = 2.27682e-16 flow_rate= [ added_flows = 0 max_flow_adds_per_second = 0 min_flow_adds_per_second = 0 deleted_flows = 0 max_flow_deletes_per_second = 0 min_flow_deletes_per_second = 0 active_flows = 0 hold_flows = 0 ] phy_band_in_bps= [ [ _iter260->first = ens6 _iter260->second = 0, ] ] phy_band_out_bps= [ [ _iter261->first = ens6 _iter261->second = 42, ] ] raw_phy_if_stats= [ [ _iter262->first = ens6 [ in_pkts = 10 in_bytes = 700 out_pkts = 178 out_bytes = 7672 ], ] ] raw_phy_if_drop_stats= [ [ _iter263->first = ens6 [ ds_discard = 0 ds_pull = 0 ds_invalid_if = 0 ds_invalid_arp = 0 ds_trap_no_if = 0 ds_nowhere_to_go = 0 ds_flow_queue_limit_exceeded = 0 ds_flow_no_memory = 0 ds_flow_invalid_protocol = 0 ds_flow_nat_no_rflow = 0 ds_flow_action_drop = 0 ds_flow_action_invalid = 0 ds_flow_unusable = 0 ds_flow_table_full = 0 ds_interface_tx_discard = 0 ds_interface_drop = 0 ds_duplicated = 0 ds_push = 0 ds_ttl_exceeded = 0 ds_invalid_nh = 0 ds_invalid_label = 0 ds_invalid_protocol = 0 ds_interface_rx_discard = 0 ds_invalid_mcast_source = 0 ds_head_alloc_fail = 0 ds_pcow_fail = 0 ds_mcast_clone_fail = 0 ds_rewrite_fail = 0 ds_misc = 0 ds_invalid_packet = 0 ds_cksum_err = 0 ds_no_fmd = 0 ds_invalid_vnid = 0 ds_frag_err = 0 ds_invalid_source = 0 ds_mcast_df_bit = 0 ds_l2_no_route = 0 ds_vlan_fwd_tx = 0 ds_vlan_fwd_enq = 0 ds_drop_new_flow = 0 ds_flow_evict = 0 ds_trap_original = 0 ds_no_memory = 0 ds_fragment_queue_fail = 0 ds_no_frag_entry = 0 ds_icmp_error = 0 ds_clone_fail = 0 ds_drop_pkts = 0 ], ] ] ] 2025-09-01 Mon 02:12:24:420.959 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011152410112, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: VrouterStats 2025-09-01 Mon 02:12:24:420.991 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011152410112, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:24:421.081 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011152410112, Pid 113882]: SANDESH: Send FAILED: 1756692744420312 [SYS_NOTICE]: VrouterControlStatsTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3993-1 raw_xmpp_stats= [ [ _iter306->first = 10.20.0.14 [ reconnects = 0 in_msgs = 0 out_msgs = 0 ], ] ] raw_ifmap_stats= [ [ _iter307->first = link_delete_parse_errors _iter307->second = 0, _iter307->first = link_update_parse_errors _iter307->second = 0, _iter307->first = node_delete_parse_errors _iter307->second = 0, _iter307->first = node_update_parse_errors _iter307->second = 0, ] ] raw_rt_table_size= [ [ _iter308->first = default-domain:default-project:ip-fabric:__default__ [ inet4_unicast = 4 inet4_multicast = 0 evpn = 0 bridge = 2 inet6_unicast = 2 ], _iter308->first = default-domain:default-project:ip-fabric:ip-fabric [ inet4_unicast = 2 inet4_multicast = 0 evpn = 1 bridge = 2 inet6_unicast = 0 ], ] ] ] 2025-09-01 Mon 02:12:24:421.102 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011152410112, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: VrouterControlStatsTrace 2025-09-01 Mon 02:12:25:431.090 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011135616512, Pid 113882]: Processing scm::EvIdleHoldTimerExpired in state Idle 2025-09-01 Mon 02:12:25:431.386 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011135616512, Pid 113882]: Connect : Start Connect timer 10.0.0.56:8086 2025-09-01 Mon 02:12:25:431.425 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011171994240, Pid 113882]: -(-1) OnSessionEvent TCP Connect Failed 2025-09-01 Mon 02:12:25:431.469 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011171994240, Pid 113882]: -(-1):EvTcpConnectFail 2025-09-01 Mon 02:12:25:431.516 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011135616512, Pid 113882]: Processing scm::EvSandeshSend in state Connect 2025-09-01 Mon 02:12:25:431.554 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011135616512, Pid 113882]: SANDESH: Send FAILED: 1756692745431195 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-3993-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.14 server_addrs= [ [ (*_iter6) = 10.20.0.14:5269, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.14 server_addrs= [ [ (*_iter6) = 10.20.0.14:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.56:8086, ] ] status = Initializing description = Idle : EvIdleHoldTimerExpired -> Connect ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.14, XMPP:dns-server:10.20.0.14, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:12:25:431.575 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011135616512, Pid 113882]: Wrong state: Connect for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:12:25:431.585 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011135616512, Pid 113882]: Processing scm::EvTcpConnectFail in state Connect 2025-09-01 Mon 02:12:25:431.591 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011135616512, Pid 113882]: Connect : EvTcpConnectFail 2025-09-01 Mon 02:12:25:431.633 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011135616512, Pid 113882]: TCP [SYS_ERR]: TcpSessionMessageLog: Session -(-1) > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:12:25:431.666 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011135616512, Pid 113882]: Idle 2025-09-01 Mon 02:12:25:431.718 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011135616512, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:25:431.852 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011135616512, Pid 113882]: SANDESH: Send FAILED: 1756692745431495 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3993-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Connect successful_connections = 0 pid = 113882 http_port = 8085 start_time = 1756692234402667 collector_name = collector_ip = 10.0.0.56:8086 collector_list= [ [ (*_iter40) = 10.0.0.56:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ComputeCpuStateTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 8 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 47 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 78 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 120 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterControlStatsTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 17 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 25 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 172 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 44 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 167, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 352, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:12:25:431.871 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011135616512, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:12:25:431.880 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011135616512, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:25:431.895 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011135616512, Pid 113882]: SANDESH: Send FAILED: 1756692745431611 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-3993-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.14 server_addrs= [ [ (*_iter6) = 10.20.0.14:5269, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.14 server_addrs= [ [ (*_iter6) = 10.20.0.14:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.14, XMPP:dns-server:10.20.0.14, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:12:25:431.901 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011135616512, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:12:25:431.912 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011135616512, Pid 113882]: Processing scm::EvTcpDeleteSession in state Idle 2025-09-01 Mon 02:12:25:431.939 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011135616512, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:25:432.051 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011135616512, Pid 113882]: SANDESH: Send FAILED: 1756692745431704 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3993-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Idle successful_connections = 0 pid = 113882 http_port = 8085 start_time = 1756692234402667 collector_name = collector_ip = 10.0.0.56:8086 collector_list= [ [ (*_iter40) = 10.0.0.56:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ComputeCpuStateTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 8 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 48 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 78 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 121 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterControlStatsTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 17 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 25 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 172 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 44 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 168, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 353, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:12:25:432.378 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011135616512, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:12:35:397.252 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011171994240, Pid 113882]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.0.0.56:8085::10.0.0.56:58330 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 255 2025-09-01 Mon 02:12:38:406.739 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011171994240, Pid 113882]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.20.0.14:8085::10.20.0.14:59432 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 255 2025-09-01 Mon 02:12:38:427.550 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011171994240, Pid 113882]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.20.0.14:8085::10.20.0.14:59446 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 255 2025-09-01 Mon 02:12:42:235.872 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:42:236.005 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: SANDESH: Send FAILED: 1756692762235797 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.14 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:42:236.034 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:42:236.198 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011171994240, Pid 113882]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.14 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:12:42:236.240 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011160806912, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:42:236.371 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:12:42:236.376 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011160806912, Pid 113882]: SANDESH: Send FAILED: 1756692762236140 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.14 state = Connect event = Connect Event local_port = 45253 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:42:236.722 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011160806912, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:42:236.795 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011160806912, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:42:236.835 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011160806912, Pid 113882]: SANDESH: Send FAILED: 1756692762236510 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.14 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:42:236.855 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011160806912, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:42:236.875 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011160806912, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:42:236.902 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011160806912, Pid 113882]: SANDESH: Send FAILED: 1756692762236526 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.14 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:42:236.920 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011160806912, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:48:841.120 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011171994240, Pid 113882]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.0.0.56:8085::10.0.0.56:42740 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 255 2025-09-01 Mon 02:12:51:009.636 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011156608512, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:51:009.758 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011156608512, Pid 113882]: SANDESH: Send FAILED: 1756692771009580 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.14 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:51:009.786 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011156608512, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:51:009.952 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011156608512, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:51:009.987 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011156608512, Pid 113882]: SANDESH: Send FAILED: 1756692771009936 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.14 state = Connect event = Connect Event local_port = 38819 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:51:010.006 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011156608512, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:51:010.032 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011171994240, Pid 113882]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.14 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:12:51:010.230 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011156608512, Pid 113882]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:12:51:010.327 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011135616512, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:51:010.419 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011135616512, Pid 113882]: SANDESH: Send FAILED: 1756692771010297 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.14 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:51:010.467 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011135616512, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:51:010.507 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011135616512, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:51:010.537 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011135616512, Pid 113882]: SANDESH: Send FAILED: 1756692771010311 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.14 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:51:010.556 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011135616512, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:54:413.933 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011160806912, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:54:414.158 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011160806912, Pid 113882]: SANDESH: Send FAILED: 1756692774413608 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3993-1:Compute:contrail-vrouter-agent:0 sm_queue_count = 0 sm_stats= [ ev_stats= [ [ [ event = scm::EvCollectorUpdate enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvIdleHoldTimerExpired enqueues = 21 dequeues = 21 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvSandeshSend enqueues = 364 dequeues = 364 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvStart enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpConnectFail enqueues = 22 dequeues = 22 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpDeleteSession enqueues = 22 dequeues = 22 enqueue_fails = 0 dequeue_fails = 0 ], [ event = enqueues = 431 dequeues = 431 enqueue_fails = 0 dequeue_fails = 0 ], ] ] state = Idle state_since = 1756692745431665 last_state = Idle last_event = scm::EvSandeshSend last_event_at = 1756692771010503 ] max_sm_queue_count = 5 ] 2025-09-01 Mon 02:12:54:414.234 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011160806912, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:12:54:414.274 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011160806912, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:54:414.672 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011160806912, Pid 113882]: SANDESH: Send FAILED: 1756692774413810 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3993-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Idle successful_connections = 0 pid = 113882 http_port = 8085 start_time = 1756692234402667 collector_name = collector_ip = 10.0.0.56:8086 collector_list= [ [ (*_iter40) = 10.0.0.56:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ComputeCpuStateTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 8 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 49 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 80 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 127 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterControlStatsTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 17 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 25 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 180 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 46 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 176, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 364, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:12:54:414.753 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011160806912, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:12:54:420.797 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011152410112, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:54:420.993 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011152410112, Pid 113882]: SANDESH: Send FAILED: 1756692774420708 [SYS_NOTICE]: VrouterStats: data= [ name = cn-jenkins-deploy-platform-ansible-os-3993-1 cpu_info= [ num_cpu = 8 sys_mem_info= [ total = 32 used = 0 free = 0 buffers = 0 cached = 3882065592 node_type = ] meminfo= [ virt = 32598 peakvirt = 0 res = 0 ] cpuload= [ one_min_avg = 0.14375 five_min_avg = 0.18625 fifteen_min_avg = 0.22125 ] cpu_share = 0.489583 ] ] 2025-09-01 Mon 02:12:54:421.057 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011152410112, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: VrouterStats 2025-09-01 Mon 02:12:54:421.088 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011152410112, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:54:421.160 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011152410112, Pid 113882]: SANDESH: Send FAILED: 1756692774420739 [SYS_NOTICE]: ComputeCpuStateTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3993-1 cpu_info= [ [ [ mem_virt = 32598 cpu_share = 0.489583 used_sys_mem = 412901704 one_min_cpuload = 0.14375 mem_res = 0 ], ] ] ] 2025-09-01 Mon 02:12:54:421.207 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011152410112, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: ComputeCpuStateTrace 2025-09-01 Mon 02:12:54:421.232 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011152410112, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:54:421.621 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011152410112, Pid 113882]: SANDESH: Send FAILED: 1756692774420789 [SYS_NOTICE]: VrouterStats: data= [ name = cn-jenkins-deploy-platform-ansible-os-3993-1 in_tpkts = 181 in_bytes = 7798 out_tpkts = 184 out_bytes = 8008 raw_vhost_stats= [ name = vhost0 in_pkts = 181 in_bytes = 7798 out_pkts = 184 out_bytes = 8008 speed = -1 duplexity = -1 ] raw_vhost_drop_stats= [ ds_discard = 0 ds_pull = 0 ds_invalid_if = 0 ds_invalid_arp = 0 ds_trap_no_if = 0 ds_nowhere_to_go = 0 ds_flow_queue_limit_exceeded = 0 ds_flow_no_memory = 0 ds_flow_invalid_protocol = 0 ds_flow_nat_no_rflow = 0 ds_flow_action_drop = 0 ds_flow_action_invalid = 0 ds_flow_unusable = 0 ds_flow_table_full = 0 ds_interface_tx_discard = 0 ds_interface_drop = 0 ds_duplicated = 0 ds_push = 0 ds_ttl_exceeded = 0 ds_invalid_nh = 0 ds_invalid_label = 0 ds_invalid_protocol = 0 ds_interface_rx_discard = 0 ds_invalid_mcast_source = 0 ds_head_alloc_fail = 0 ds_pcow_fail = 0 ds_mcast_clone_fail = 0 ds_rewrite_fail = 0 ds_misc = 0 ds_invalid_packet = 0 ds_cksum_err = 0 ds_no_fmd = 0 ds_invalid_vnid = 0 ds_frag_err = 0 ds_invalid_source = 0 ds_mcast_df_bit = 0 ds_l2_no_route = 0 ds_vlan_fwd_tx = 0 ds_vlan_fwd_enq = 0 ds_drop_new_flow = 0 ds_flow_evict = 0 ds_trap_original = 0 ds_no_memory = 0 ds_fragment_queue_fail = 0 ds_no_frag_entry = 0 ds_icmp_error = 0 ds_clone_fail = 0 ds_drop_pkts = 0 ] raw_drop_stats= [ ds_discard = 0 ds_pull = 0 ds_invalid_if = 0 ds_invalid_arp = 0 ds_trap_no_if = 0 ds_nowhere_to_go = 0 ds_flow_queue_limit_exceeded = 0 ds_flow_no_memory = 0 ds_flow_invalid_protocol = 0 ds_flow_nat_no_rflow = 0 ds_flow_action_drop = 0 ds_flow_action_invalid = 0 ds_flow_unusable = 0 ds_flow_table_full = 0 ds_interface_tx_discard = 0 ds_interface_drop = 0 ds_duplicated = 0 ds_push = 0 ds_ttl_exceeded = 0 ds_invalid_nh = 0 ds_invalid_label = 0 ds_invalid_protocol = 0 ds_interface_rx_discard = 0 ds_invalid_mcast_source = 0 ds_head_alloc_fail = 0 ds_pcow_fail = 0 ds_mcast_clone_fail = 0 ds_rewrite_fail = 0 ds_misc = 0 ds_invalid_packet = 0 ds_cksum_err = 0 ds_no_fmd = 0 ds_invalid_vnid = 0 ds_frag_err = 0 ds_invalid_source = 0 ds_mcast_df_bit = 0 ds_l2_no_route = 0 ds_vlan_fwd_tx = 0 ds_vlan_fwd_enq = 0 ds_drop_new_flow = 0 ds_flow_evict = 0 ds_trap_original = 0 ds_no_memory = 0 ds_fragment_queue_fail = 0 ds_no_frag_entry = 0 ds_icmp_error = 0 ds_clone_fail = 0 ds_drop_pkts = 0 ] flow_rate= [ added_flows = 0 max_flow_adds_per_second = 0 min_flow_adds_per_second = 0 deleted_flows = 0 max_flow_deletes_per_second = 0 min_flow_deletes_per_second = 0 active_flows = 0 hold_flows = 0 ] raw_phy_if_stats= [ [ _iter262->first = ens6 [ in_pkts = 10 in_bytes = 700 out_pkts = 181 out_bytes = 7798 ], ] ] raw_phy_if_drop_stats= [ [ _iter263->first = ens6 [ ds_discard = 0 ds_pull = 0 ds_invalid_if = 0 ds_invalid_arp = 0 ds_trap_no_if = 0 ds_nowhere_to_go = 0 ds_flow_queue_limit_exceeded = 0 ds_flow_no_memory = 0 ds_flow_invalid_protocol = 0 ds_flow_nat_no_rflow = 0 ds_flow_action_drop = 0 ds_flow_action_invalid = 0 ds_flow_unusable = 0 ds_flow_table_full = 0 ds_interface_tx_discard = 0 ds_interface_drop = 0 ds_duplicated = 0 ds_push = 0 ds_ttl_exceeded = 0 ds_invalid_nh = 0 ds_invalid_label = 0 ds_invalid_protocol = 0 ds_interface_rx_discard = 0 ds_invalid_mcast_source = 0 ds_head_alloc_fail = 0 ds_pcow_fail = 0 ds_mcast_clone_fail = 0 ds_rewrite_fail = 0 ds_misc = 0 ds_invalid_packet = 0 ds_cksum_err = 0 ds_no_fmd = 0 ds_invalid_vnid = 0 ds_frag_err = 0 ds_invalid_source = 0 ds_mcast_df_bit = 0 ds_l2_no_route = 0 ds_vlan_fwd_tx = 0 ds_vlan_fwd_enq = 0 ds_drop_new_flow = 0 ds_flow_evict = 0 ds_trap_original = 0 ds_no_memory = 0 ds_fragment_queue_fail = 0 ds_no_frag_entry = 0 ds_icmp_error = 0 ds_clone_fail = 0 ds_drop_pkts = 0 ], ] ] ] 2025-09-01 Mon 02:12:54:421.673 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011152410112, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: VrouterStats 2025-09-01 Mon 02:12:54:421.698 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011152410112, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:54:421.774 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011152410112, Pid 113882]: SANDESH: Send FAILED: 1756692774420954 [SYS_NOTICE]: VrouterControlStatsTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3993-1 raw_xmpp_stats= [ [ _iter306->first = 10.20.0.14 [ reconnects = 0 in_msgs = 0 out_msgs = 0 ], ] ] raw_ifmap_stats= [ [ _iter307->first = link_delete_parse_errors _iter307->second = 0, _iter307->first = link_update_parse_errors _iter307->second = 0, _iter307->first = node_delete_parse_errors _iter307->second = 0, _iter307->first = node_update_parse_errors _iter307->second = 0, ] ] raw_rt_table_size= [ [ _iter308->first = default-domain:default-project:ip-fabric:__default__ [ inet4_unicast = 4 inet4_multicast = 0 evpn = 0 bridge = 2 inet6_unicast = 2 ], _iter308->first = default-domain:default-project:ip-fabric:ip-fabric [ inet4_unicast = 2 inet4_multicast = 0 evpn = 1 bridge = 2 inet6_unicast = 0 ], ] ] ] 2025-09-01 Mon 02:12:54:421.794 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011152410112, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: VrouterControlStatsTrace 2025-09-01 Mon 02:12:55:431.804 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011152410112, Pid 113882]: Processing scm::EvIdleHoldTimerExpired in state Idle 2025-09-01 Mon 02:12:55:432.060 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011152410112, Pid 113882]: Connect : Start Connect timer 10.0.0.56:8086 2025-09-01 Mon 02:12:55:432.144 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011171994240, Pid 113882]: -(-1) OnSessionEvent TCP Connect Failed 2025-09-01 Mon 02:12:55:432.166 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011152410112, Pid 113882]: Processing scm::EvSandeshSend in state Connect 2025-09-01 Mon 02:12:55:432.257 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011171994240, Pid 113882]: -(-1):EvTcpConnectFail 2025-09-01 Mon 02:12:55:432.292 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011152410112, Pid 113882]: SANDESH: Send FAILED: 1756692775431887 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-3993-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.14 server_addrs= [ [ (*_iter6) = 10.20.0.14:5269, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.14 server_addrs= [ [ (*_iter6) = 10.20.0.14:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.56:8086, ] ] status = Initializing description = Idle : EvIdleHoldTimerExpired -> Connect ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.14, XMPP:dns-server:10.20.0.14, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:12:55:432.344 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011152410112, Pid 113882]: Wrong state: Connect for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:12:55:432.354 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011152410112, Pid 113882]: Processing scm::EvSandeshSend in state Connect 2025-09-01 Mon 02:12:55:432.507 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011152410112, Pid 113882]: SANDESH: Send FAILED: 1756692775432147 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3993-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Connect successful_connections = 0 pid = 113882 http_port = 8085 start_time = 1756692234402667 collector_name = collector_ip = 10.0.0.56:8086 collector_list= [ [ (*_iter40) = 10.0.0.56:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ComputeCpuStateTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 9 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 49 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 82 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 127 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterControlStatsTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 18 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 27 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 180 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 46 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 176, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 370, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:12:55:432.814 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011152410112, Pid 113882]: Wrong state: Connect for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:12:55:432.843 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011152410112, Pid 113882]: Processing scm::EvTcpConnectFail in state Connect 2025-09-01 Mon 02:12:55:432.850 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011152410112, Pid 113882]: Connect : EvTcpConnectFail 2025-09-01 Mon 02:12:55:432.904 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011152410112, Pid 113882]: TCP [SYS_ERR]: TcpSessionMessageLog: Session -(-1) > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:12:55:432.933 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011152410112, Pid 113882]: Idle 2025-09-01 Mon 02:12:55:432.998 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011152410112, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:55:433.019 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011152410112, Pid 113882]: SANDESH: Send FAILED: 1756692775432875 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-3993-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.14 server_addrs= [ [ (*_iter6) = 10.20.0.14:5269, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.14 server_addrs= [ [ (*_iter6) = 10.20.0.14:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.14, XMPP:dns-server:10.20.0.14, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:12:55:433.026 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011152410112, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:12:55:433.038 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011152410112, Pid 113882]: Processing scm::EvTcpDeleteSession in state Idle 2025-09-01 Mon 02:12:55:433.068 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011152410112, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:55:433.215 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011152410112, Pid 113882]: SANDESH: Send FAILED: 1756692775432985 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3993-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Idle successful_connections = 0 pid = 113882 http_port = 8085 start_time = 1756692234402667 collector_name = collector_ip = 10.0.0.56:8086 collector_list= [ [ (*_iter40) = 10.0.0.56:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ComputeCpuStateTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 9 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 50 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 83 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 128 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterControlStatsTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 18 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 27 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 180 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 46 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 177, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 372, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:12:55:433.238 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011152410112, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:13:01:738.000 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011171994240, Pid 113882]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.0.0.56:8085::10.0.0.56:57190 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 255 2025-09-01 Mon 02:13:11:126.811 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011152410112, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:11:126.916 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011152410112, Pid 113882]: SANDESH: Send FAILED: 1756692791126752 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.14 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:11:126.939 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011152410112, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:11:127.094 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:11:127.101 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011171994240, Pid 113882]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.14 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:13:11:127.196 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: SANDESH: Send FAILED: 1756692791127043 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.14 state = Connect event = Connect Event local_port = 38107 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:11:127.281 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:11:127.301 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011160806912, Pid 113882]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:13:11:127.474 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011135616512, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:11:127.583 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011135616512, Pid 113882]: SANDESH: Send FAILED: 1756692791127358 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.14 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:11:127.613 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011135616512, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:11:127.641 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011135616512, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:11:127.674 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011135616512, Pid 113882]: SANDESH: Send FAILED: 1756692791127389 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.14 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:11:127.721 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011135616512, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:14:784.321 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011171994240, Pid 113882]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.0.0.56:8085::10.0.0.56:36536 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 255 2025-09-01 Mon 02:13:22:060.706 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011139814912, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:22:060.782 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011139814912, Pid 113882]: SANDESH: Send FAILED: 1756692802060660 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.14 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:22:060.800 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011139814912, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:22:061.103 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011171994240, Pid 113882]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.14 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:13:22:061.227 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011152410112, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:22:061.256 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011135616512, Pid 113882]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:13:22:061.335 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011152410112, Pid 113882]: SANDESH: Send FAILED: 1756692802061083 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.14 state = Connect event = Connect Event local_port = 40365 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:22:061.636 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011152410112, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:22:061.679 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011152410112, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:22:061.739 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011152410112, Pid 113882]: SANDESH: Send FAILED: 1756692802061428 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.14 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:22:061.769 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011152410112, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:22:061.800 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011152410112, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:22:061.831 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011152410112, Pid 113882]: SANDESH: Send FAILED: 1756692802061441 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.14 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:22:061.849 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011152410112, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:24:414.560 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011135616512, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:24:414.862 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011135616512, Pid 113882]: SANDESH: Send FAILED: 1756692804414120 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3993-1:Compute:contrail-vrouter-agent:0 sm_queue_count = 0 sm_stats= [ ev_stats= [ [ [ event = scm::EvCollectorUpdate enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvIdleHoldTimerExpired enqueues = 22 dequeues = 22 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvSandeshSend enqueues = 382 dequeues = 382 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvStart enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpConnectFail enqueues = 23 dequeues = 23 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpDeleteSession enqueues = 23 dequeues = 23 enqueue_fails = 0 dequeue_fails = 0 ], [ event = enqueues = 452 dequeues = 452 enqueue_fails = 0 dequeue_fails = 0 ], ] ] state = Idle state_since = 1756692775432932 last_state = Idle last_event = scm::EvSandeshSend last_event_at = 1756692802061796 ] max_sm_queue_count = 5 ] 2025-09-01 Mon 02:13:24:414.918 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011135616512, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:13:24:414.959 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011135616512, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:24:415.770 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011135616512, Pid 113882]: SANDESH: Send FAILED: 1756692804414424 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3993-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Idle successful_connections = 0 pid = 113882 http_port = 8085 start_time = 1756692234402667 collector_name = collector_ip = 10.0.0.56:8086 collector_list= [ [ (*_iter40) = 10.0.0.56:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ComputeCpuStateTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 9 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 51 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 84 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 132 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterControlStatsTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 18 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 27 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 188 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 48 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 183, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 382, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:13:24:415.879 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011135616512, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:13:24:421.296 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011152410112, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:24:421.724 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011152410112, Pid 113882]: SANDESH: Send FAILED: 1756692804421233 [SYS_NOTICE]: VrouterStats: data= [ name = cn-jenkins-deploy-platform-ansible-os-3993-1 in_tpkts = 184 in_bytes = 7924 out_tpkts = 187 out_bytes = 8134 exception_packets = 13 exception_packets_dropped = 13 raw_vhost_stats= [ name = vhost0 in_pkts = 184 in_bytes = 7924 out_pkts = 187 out_bytes = 8134 speed = -1 duplexity = -1 ] raw_vhost_drop_stats= [ ds_discard = 0 ds_pull = 0 ds_invalid_if = 0 ds_invalid_arp = 0 ds_trap_no_if = 0 ds_nowhere_to_go = 0 ds_flow_queue_limit_exceeded = 0 ds_flow_no_memory = 0 ds_flow_invalid_protocol = 0 ds_flow_nat_no_rflow = 0 ds_flow_action_drop = 0 ds_flow_action_invalid = 0 ds_flow_unusable = 0 ds_flow_table_full = 0 ds_interface_tx_discard = 0 ds_interface_drop = 0 ds_duplicated = 0 ds_push = 0 ds_ttl_exceeded = 0 ds_invalid_nh = 0 ds_invalid_label = 0 ds_invalid_protocol = 0 ds_interface_rx_discard = 0 ds_invalid_mcast_source = 0 ds_head_alloc_fail = 0 ds_pcow_fail = 0 ds_mcast_clone_fail = 0 ds_rewrite_fail = 0 ds_misc = 0 ds_invalid_packet = 0 ds_cksum_err = 0 ds_no_fmd = 0 ds_invalid_vnid = 0 ds_frag_err = 0 ds_invalid_source = 0 ds_mcast_df_bit = 0 ds_l2_no_route = 0 ds_vlan_fwd_tx = 0 ds_vlan_fwd_enq = 0 ds_drop_new_flow = 0 ds_flow_evict = 0 ds_trap_original = 0 ds_no_memory = 0 ds_fragment_queue_fail = 0 ds_no_frag_entry = 0 ds_icmp_error = 0 ds_clone_fail = 0 ds_drop_pkts = 0 ] raw_drop_stats= [ ds_discard = 0 ds_pull = 0 ds_invalid_if = 0 ds_invalid_arp = 0 ds_trap_no_if = 0 ds_nowhere_to_go = 0 ds_flow_queue_limit_exceeded = 0 ds_flow_no_memory = 0 ds_flow_invalid_protocol = 0 ds_flow_nat_no_rflow = 0 ds_flow_action_drop = 0 ds_flow_action_invalid = 0 ds_flow_unusable = 0 ds_flow_table_full = 0 ds_interface_tx_discard = 0 ds_interface_drop = 0 ds_duplicated = 0 ds_push = 0 ds_ttl_exceeded = 0 ds_invalid_nh = 0 ds_invalid_label = 0 ds_invalid_protocol = 0 ds_interface_rx_discard = 0 ds_invalid_mcast_source = 0 ds_head_alloc_fail = 0 ds_pcow_fail = 0 ds_mcast_clone_fail = 0 ds_rewrite_fail = 0 ds_misc = 0 ds_invalid_packet = 0 ds_cksum_err = 0 ds_no_fmd = 0 ds_invalid_vnid = 0 ds_frag_err = 0 ds_invalid_source = 0 ds_mcast_df_bit = 0 ds_l2_no_route = 0 ds_vlan_fwd_tx = 0 ds_vlan_fwd_enq = 0 ds_drop_new_flow = 0 ds_flow_evict = 0 ds_trap_original = 0 ds_no_memory = 0 ds_fragment_queue_fail = 0 ds_no_frag_entry = 0 ds_icmp_error = 0 ds_clone_fail = 0 ds_drop_pkts = 0 ] total_out_bandwidth_utilization = 1.78893e-16 flow_rate= [ added_flows = 0 max_flow_adds_per_second = 0 min_flow_adds_per_second = 0 deleted_flows = 0 max_flow_deletes_per_second = 0 min_flow_deletes_per_second = 0 active_flows = 0 hold_flows = 0 ] phy_band_in_bps= [ [ _iter260->first = ens6 _iter260->second = 0, ] ] phy_band_out_bps= [ [ _iter261->first = ens6 _iter261->second = 33, ] ] raw_phy_if_stats= [ [ _iter262->first = ens6 [ in_pkts = 10 in_bytes = 700 out_pkts = 184 out_bytes = 7924 ], ] ] raw_phy_if_drop_stats= [ [ _iter263->first = ens6 [ ds_discard = 0 ds_pull = 0 ds_invalid_if = 0 ds_invalid_arp = 0 ds_trap_no_if = 0 ds_nowhere_to_go = 0 ds_flow_queue_limit_exceeded = 0 ds_flow_no_memory = 0 ds_flow_invalid_protocol = 0 ds_flow_nat_no_rflow = 0 ds_flow_action_drop = 0 ds_flow_action_invalid = 0 ds_flow_unusable = 0 ds_flow_table_full = 0 ds_interface_tx_discard = 0 ds_interface_drop = 0 ds_duplicated = 0 ds_push = 0 ds_ttl_exceeded = 0 ds_invalid_nh = 0 ds_invalid_label = 0 ds_invalid_protocol = 0 ds_interface_rx_discard = 0 ds_invalid_mcast_source = 0 ds_head_alloc_fail = 0 ds_pcow_fail = 0 ds_mcast_clone_fail = 0 ds_rewrite_fail = 0 ds_misc = 0 ds_invalid_packet = 0 ds_cksum_err = 0 ds_no_fmd = 0 ds_invalid_vnid = 0 ds_frag_err = 0 ds_invalid_source = 0 ds_mcast_df_bit = 0 ds_l2_no_route = 0 ds_vlan_fwd_tx = 0 ds_vlan_fwd_enq = 0 ds_drop_new_flow = 0 ds_flow_evict = 0 ds_trap_original = 0 ds_no_memory = 0 ds_fragment_queue_fail = 0 ds_no_frag_entry = 0 ds_icmp_error = 0 ds_clone_fail = 0 ds_drop_pkts = 0 ], ] ] ] 2025-09-01 Mon 02:13:24:421.794 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011152410112, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: VrouterStats 2025-09-01 Mon 02:13:24:421.829 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011152410112, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:24:421.906 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011152410112, Pid 113882]: SANDESH: Send FAILED: 1756692804421329 [SYS_NOTICE]: VrouterControlStatsTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3993-1 raw_xmpp_stats= [ [ _iter306->first = 10.20.0.14 [ reconnects = 0 in_msgs = 0 out_msgs = 0 ], ] ] raw_ifmap_stats= [ [ _iter307->first = link_delete_parse_errors _iter307->second = 0, _iter307->first = link_update_parse_errors _iter307->second = 0, _iter307->first = node_delete_parse_errors _iter307->second = 0, _iter307->first = node_update_parse_errors _iter307->second = 0, ] ] raw_rt_table_size= [ [ _iter308->first = default-domain:default-project:ip-fabric:__default__ [ inet4_unicast = 4 inet4_multicast = 0 evpn = 0 bridge = 2 inet6_unicast = 2 ], _iter308->first = default-domain:default-project:ip-fabric:ip-fabric [ inet4_unicast = 2 inet4_multicast = 0 evpn = 1 bridge = 2 inet6_unicast = 0 ], ] ] ] 2025-09-01 Mon 02:13:24:421.936 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011152410112, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: VrouterControlStatsTrace 2025-09-01 Mon 02:13:25:433.158 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011139814912, Pid 113882]: Processing scm::EvIdleHoldTimerExpired in state Idle 2025-09-01 Mon 02:13:25:433.490 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011139814912, Pid 113882]: Connect : Start Connect timer 10.0.0.56:8086 2025-09-01 Mon 02:13:25:433.578 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011171994240, Pid 113882]: -(-1) OnSessionEvent TCP Connect Failed 2025-09-01 Mon 02:13:25:433.619 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011139814912, Pid 113882]: Processing scm::EvSandeshSend in state Connect 2025-09-01 Mon 02:13:25:433.682 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011171994240, Pid 113882]: -(-1):EvTcpConnectFail 2025-09-01 Mon 02:13:25:433.739 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011139814912, Pid 113882]: SANDESH: Send FAILED: 1756692805433279 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-3993-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.14 server_addrs= [ [ (*_iter6) = 10.20.0.14:5269, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.14 server_addrs= [ [ (*_iter6) = 10.20.0.14:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.56:8086, ] ] status = Initializing description = Idle : EvIdleHoldTimerExpired -> Connect ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.14, XMPP:dns-server:10.20.0.14, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:13:25:433.751 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011139814912, Pid 113882]: Wrong state: Connect for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:13:25:433.761 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011139814912, Pid 113882]: Processing scm::EvSandeshSend in state Connect 2025-09-01 Mon 02:13:25:433.887 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011139814912, Pid 113882]: SANDESH: Send FAILED: 1756692805433590 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3993-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Connect successful_connections = 0 pid = 113882 http_port = 8085 start_time = 1756692234402667 collector_name = collector_ip = 10.0.0.56:8086 collector_list= [ [ (*_iter40) = 10.0.0.56:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ComputeCpuStateTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 9 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 51 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 86 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 132 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterControlStatsTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 19 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 28 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 188 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 48 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 183, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 386, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:13:25:433.914 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011139814912, Pid 113882]: Wrong state: Connect for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:13:25:433.924 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011139814912, Pid 113882]: Processing scm::EvTcpConnectFail in state Connect 2025-09-01 Mon 02:13:25:433.930 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011139814912, Pid 113882]: Connect : EvTcpConnectFail 2025-09-01 Mon 02:13:25:433.981 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011139814912, Pid 113882]: TCP [SYS_ERR]: TcpSessionMessageLog: Session -(-1) > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:13:25:434.020 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011139814912, Pid 113882]: Idle 2025-09-01 Mon 02:13:25:434.096 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011139814912, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:25:434.114 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011139814912, Pid 113882]: SANDESH: Send FAILED: 1756692805433952 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-3993-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.14 server_addrs= [ [ (*_iter6) = 10.20.0.14:5269, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.14 server_addrs= [ [ (*_iter6) = 10.20.0.14:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.14, XMPP:dns-server:10.20.0.14, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:13:25:434.120 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011139814912, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:13:25:434.127 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011139814912, Pid 113882]: Processing scm::EvTcpDeleteSession in state Idle 2025-09-01 Mon 02:13:25:434.157 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011139814912, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:25:434.274 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011139814912, Pid 113882]: SANDESH: Send FAILED: 1756692805434081 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3993-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Idle successful_connections = 0 pid = 113882 http_port = 8085 start_time = 1756692234402667 collector_name = collector_ip = 10.0.0.56:8086 collector_list= [ [ (*_iter40) = 10.0.0.56:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ComputeCpuStateTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 9 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 52 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 87 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 133 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterControlStatsTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 19 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 28 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 188 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 48 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 184, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 388, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:13:25:434.304 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011139814912, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:13:27:768.343 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011171994240, Pid 113882]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.0.0.56:8085::10.0.0.56:44356 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 255 2025-09-01 Mon 02:13:39:421.805 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011171994240, Pid 113882]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.20.0.14:8085::10.20.0.14:50142 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 255 2025-09-01 Mon 02:13:39:438.101 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011171994240, Pid 113882]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.20.0.14:8085::10.20.0.14:50148 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 255 2025-09-01 Mon 02:13:40:827.698 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:40:827.810 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: SANDESH: Send FAILED: 1756692820827644 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.14 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:40:827.838 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:40:828.017 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011171994240, Pid 113882]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.14 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:13:40:828.057 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011156608512, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:40:828.150 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011156608512, Pid 113882]: SANDESH: Send FAILED: 1756692820827990 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.14 state = Connect event = Connect Event local_port = 37105 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:40:828.199 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011156608512, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:40:828.228 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011152410112, Pid 113882]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:13:40:828.471 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011139814912, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:40:828.517 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011139814912, Pid 113882]: SANDESH: Send FAILED: 1756692820828329 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.14 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:40:828.536 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011139814912, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:40:828.559 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011139814912, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:40:828.586 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011139814912, Pid 113882]: SANDESH: Send FAILED: 1756692820828405 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.14 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:40:828.603 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011139814912, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:40:905.030 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011171994240, Pid 113882]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.0.0.56:8085::10.0.0.56:55118 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 255 2025-09-01 Mon 02:13:51:221.883 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011148211712, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:51:222.017 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011148211712, Pid 113882]: SANDESH: Send FAILED: 1756692831221825 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.14 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:51:222.057 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011148211712, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:51:222.287 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:51:222.292 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011171994240, Pid 113882]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.14 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:13:51:222.413 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: SANDESH: Send FAILED: 1756692831222206 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.14 state = Connect event = Connect Event local_port = 37787 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:51:222.577 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:51:222.608 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011148211712, Pid 113882]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:13:51:222.796 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:51:222.869 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: SANDESH: Send FAILED: 1756692831222688 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.14 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:51:222.890 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:51:222.912 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:51:222.939 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: SANDESH: Send FAILED: 1756692831222735 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.14 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:51:222.956 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:54:225.397 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011171994240, Pid 113882]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.0.0.56:8085::10.0.0.56:47256 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 255 2025-09-01 Mon 02:13:54:414.949 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011160806912, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:54:415.126 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011160806912, Pid 113882]: SANDESH: Send FAILED: 1756692834414641 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3993-1:Compute:contrail-vrouter-agent:0 sm_queue_count = 0 sm_stats= [ ev_stats= [ [ [ event = scm::EvCollectorUpdate enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvIdleHoldTimerExpired enqueues = 23 dequeues = 23 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvSandeshSend enqueues = 398 dequeues = 398 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvStart enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpConnectFail enqueues = 24 dequeues = 24 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpDeleteSession enqueues = 24 dequeues = 24 enqueue_fails = 0 dequeue_fails = 0 ], [ event = enqueues = 471 dequeues = 471 enqueue_fails = 0 dequeue_fails = 0 ], ] ] state = Idle state_since = 1756692805434018 last_state = Idle last_event = scm::EvSandeshSend last_event_at = 1756692831222909 ] max_sm_queue_count = 5 ] 2025-09-01 Mon 02:13:54:415.201 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011160806912, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:13:54:415.246 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011160806912, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:54:415.641 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011160806912, Pid 113882]: SANDESH: Send FAILED: 1756692834414844 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3993-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Idle successful_connections = 0 pid = 113882 http_port = 8085 start_time = 1756692234402667 collector_name = collector_ip = 10.0.0.56:8086 collector_list= [ [ (*_iter40) = 10.0.0.56:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ComputeCpuStateTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 9 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 53 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 88 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 140 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterControlStatsTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 19 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 28 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 196 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 50 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 193, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 398, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:13:54:415.688 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011160806912, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:13:54:421.655 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:54:421.772 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: SANDESH: Send FAILED: 1756692834421599 [SYS_NOTICE]: VrouterStats: data= [ name = cn-jenkins-deploy-platform-ansible-os-3993-1 cpu_info= [ num_cpu = 8 sys_mem_info= [ total = 32 used = 0 free = 0 buffers = 0 cached = 3882065592 node_type = ] meminfo= [ virt = 32598 peakvirt = 0 res = 0 ] cpuload= [ one_min_avg = 0.19375 five_min_avg = 0.19 fifteen_min_avg = 0.22 ] cpu_share = 0.491667 ] ] 2025-09-01 Mon 02:13:54:421.793 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: VrouterStats 2025-09-01 Mon 02:13:54:421.814 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:54:421.852 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: SANDESH: Send FAILED: 1756692834421629 [SYS_NOTICE]: ComputeCpuStateTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3993-1 cpu_info= [ [ [ mem_virt = 32598 cpu_share = 0.491667 used_sys_mem = 412901704 one_min_cpuload = 0.19375 mem_res = 0 ], ] ] ] 2025-09-01 Mon 02:13:54:421.868 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: ComputeCpuStateTrace 2025-09-01 Mon 02:13:54:421.885 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:54:422.208 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: SANDESH: Send FAILED: 1756692834421687 [SYS_NOTICE]: VrouterStats: data= [ name = cn-jenkins-deploy-platform-ansible-os-3993-1 in_tpkts = 187 in_bytes = 8050 out_tpkts = 190 out_bytes = 8260 raw_vhost_stats= [ name = vhost0 in_pkts = 187 in_bytes = 8050 out_pkts = 190 out_bytes = 8260 speed = -1 duplexity = -1 ] raw_vhost_drop_stats= [ ds_discard = 0 ds_pull = 0 ds_invalid_if = 0 ds_invalid_arp = 0 ds_trap_no_if = 0 ds_nowhere_to_go = 0 ds_flow_queue_limit_exceeded = 0 ds_flow_no_memory = 0 ds_flow_invalid_protocol = 0 ds_flow_nat_no_rflow = 0 ds_flow_action_drop = 0 ds_flow_action_invalid = 0 ds_flow_unusable = 0 ds_flow_table_full = 0 ds_interface_tx_discard = 0 ds_interface_drop = 0 ds_duplicated = 0 ds_push = 0 ds_ttl_exceeded = 0 ds_invalid_nh = 0 ds_invalid_label = 0 ds_invalid_protocol = 0 ds_interface_rx_discard = 0 ds_invalid_mcast_source = 0 ds_head_alloc_fail = 0 ds_pcow_fail = 0 ds_mcast_clone_fail = 0 ds_rewrite_fail = 0 ds_misc = 0 ds_invalid_packet = 0 ds_cksum_err = 0 ds_no_fmd = 0 ds_invalid_vnid = 0 ds_frag_err = 0 ds_invalid_source = 0 ds_mcast_df_bit = 0 ds_l2_no_route = 0 ds_vlan_fwd_tx = 0 ds_vlan_fwd_enq = 0 ds_drop_new_flow = 0 ds_flow_evict = 0 ds_trap_original = 0 ds_no_memory = 0 ds_fragment_queue_fail = 0 ds_no_frag_entry = 0 ds_icmp_error = 0 ds_clone_fail = 0 ds_drop_pkts = 0 ] raw_drop_stats= [ ds_discard = 0 ds_pull = 0 ds_invalid_if = 0 ds_invalid_arp = 0 ds_trap_no_if = 0 ds_nowhere_to_go = 0 ds_flow_queue_limit_exceeded = 0 ds_flow_no_memory = 0 ds_flow_invalid_protocol = 0 ds_flow_nat_no_rflow = 0 ds_flow_action_drop = 0 ds_flow_action_invalid = 0 ds_flow_unusable = 0 ds_flow_table_full = 0 ds_interface_tx_discard = 0 ds_interface_drop = 0 ds_duplicated = 0 ds_push = 0 ds_ttl_exceeded = 0 ds_invalid_nh = 0 ds_invalid_label = 0 ds_invalid_protocol = 0 ds_interface_rx_discard = 0 ds_invalid_mcast_source = 0 ds_head_alloc_fail = 0 ds_pcow_fail = 0 ds_mcast_clone_fail = 0 ds_rewrite_fail = 0 ds_misc = 0 ds_invalid_packet = 0 ds_cksum_err = 0 ds_no_fmd = 0 ds_invalid_vnid = 0 ds_frag_err = 0 ds_invalid_source = 0 ds_mcast_df_bit = 0 ds_l2_no_route = 0 ds_vlan_fwd_tx = 0 ds_vlan_fwd_enq = 0 ds_drop_new_flow = 0 ds_flow_evict = 0 ds_trap_original = 0 ds_no_memory = 0 ds_fragment_queue_fail = 0 ds_no_frag_entry = 0 ds_icmp_error = 0 ds_clone_fail = 0 ds_drop_pkts = 0 ] flow_rate= [ added_flows = 0 max_flow_adds_per_second = 0 min_flow_adds_per_second = 0 deleted_flows = 0 max_flow_deletes_per_second = 0 min_flow_deletes_per_second = 0 active_flows = 0 hold_flows = 0 ] raw_phy_if_stats= [ [ _iter262->first = ens6 [ in_pkts = 10 in_bytes = 700 out_pkts = 187 out_bytes = 8050 ], ] ] raw_phy_if_drop_stats= [ [ _iter263->first = ens6 [ ds_discard = 0 ds_pull = 0 ds_invalid_if = 0 ds_invalid_arp = 0 ds_trap_no_if = 0 ds_nowhere_to_go = 0 ds_flow_queue_limit_exceeded = 0 ds_flow_no_memory = 0 ds_flow_invalid_protocol = 0 ds_flow_nat_no_rflow = 0 ds_flow_action_drop = 0 ds_flow_action_invalid = 0 ds_flow_unusable = 0 ds_flow_table_full = 0 ds_interface_tx_discard = 0 ds_interface_drop = 0 ds_duplicated = 0 ds_push = 0 ds_ttl_exceeded = 0 ds_invalid_nh = 0 ds_invalid_label = 0 ds_invalid_protocol = 0 ds_interface_rx_discard = 0 ds_invalid_mcast_source = 0 ds_head_alloc_fail = 0 ds_pcow_fail = 0 ds_mcast_clone_fail = 0 ds_rewrite_fail = 0 ds_misc = 0 ds_invalid_packet = 0 ds_cksum_err = 0 ds_no_fmd = 0 ds_invalid_vnid = 0 ds_frag_err = 0 ds_invalid_source = 0 ds_mcast_df_bit = 0 ds_l2_no_route = 0 ds_vlan_fwd_tx = 0 ds_vlan_fwd_enq = 0 ds_drop_new_flow = 0 ds_flow_evict = 0 ds_trap_original = 0 ds_no_memory = 0 ds_fragment_queue_fail = 0 ds_no_frag_entry = 0 ds_icmp_error = 0 ds_clone_fail = 0 ds_drop_pkts = 0 ], ] ] ] 2025-09-01 Mon 02:13:54:422.260 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: VrouterStats 2025-09-01 Mon 02:13:54:422.283 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:54:422.348 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: SANDESH: Send FAILED: 1756692834421756 [SYS_NOTICE]: VrouterControlStatsTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3993-1 raw_xmpp_stats= [ [ _iter306->first = 10.20.0.14 [ reconnects = 0 in_msgs = 0 out_msgs = 0 ], ] ] raw_ifmap_stats= [ [ _iter307->first = link_delete_parse_errors _iter307->second = 0, _iter307->first = link_update_parse_errors _iter307->second = 0, _iter307->first = node_delete_parse_errors _iter307->second = 0, _iter307->first = node_update_parse_errors _iter307->second = 0, ] ] raw_rt_table_size= [ [ _iter308->first = default-domain:default-project:ip-fabric:__default__ [ inet4_unicast = 4 inet4_multicast = 0 evpn = 0 bridge = 2 inet6_unicast = 2 ], _iter308->first = default-domain:default-project:ip-fabric:ip-fabric [ inet4_unicast = 2 inet4_multicast = 0 evpn = 1 bridge = 2 inet6_unicast = 0 ], ] ] ] 2025-09-01 Mon 02:13:54:422.371 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: VrouterControlStatsTrace 2025-09-01 Mon 02:13:55:434.296 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011135616512, Pid 113882]: Processing scm::EvIdleHoldTimerExpired in state Idle 2025-09-01 Mon 02:13:55:434.845 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011135616512, Pid 113882]: Connect : Start Connect timer 10.0.0.56:8086 2025-09-01 Mon 02:13:55:434.890 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011171994240, Pid 113882]: -(-1) OnSessionEvent TCP Connect Failed 2025-09-01 Mon 02:13:55:434.958 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011171994240, Pid 113882]: -(-1):EvTcpConnectFail 2025-09-01 Mon 02:13:55:435.110 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011135616512, Pid 113882]: Processing scm::EvSandeshSend in state Connect 2025-09-01 Mon 02:13:55:435.198 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011135616512, Pid 113882]: SANDESH: Send FAILED: 1756692835434481 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-3993-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.14 server_addrs= [ [ (*_iter6) = 10.20.0.14:5269, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.14 server_addrs= [ [ (*_iter6) = 10.20.0.14:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.56:8086, ] ] status = Initializing description = Idle : EvIdleHoldTimerExpired -> Connect ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.14, XMPP:dns-server:10.20.0.14, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:13:55:435.224 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011135616512, Pid 113882]: Wrong state: Connect for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:13:55:435.251 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011135616512, Pid 113882]: Processing scm::EvTcpConnectFail in state Connect 2025-09-01 Mon 02:13:55:435.271 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011135616512, Pid 113882]: Connect : EvTcpConnectFail 2025-09-01 Mon 02:13:55:435.378 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011135616512, Pid 113882]: TCP [SYS_ERR]: TcpSessionMessageLog: Session -(-1) > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:13:55:435.447 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011135616512, Pid 113882]: Idle 2025-09-01 Mon 02:13:55:435.580 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011135616512, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:55:436.061 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011135616512, Pid 113882]: SANDESH: Send FAILED: 1756692835435058 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3993-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Connect successful_connections = 0 pid = 113882 http_port = 8085 start_time = 1756692234402667 collector_name = collector_ip = 10.0.0.56:8086 collector_list= [ [ (*_iter40) = 10.0.0.56:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ComputeCpuStateTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 10 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 53 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 90 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 140 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterControlStatsTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 20 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 30 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 196 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 50 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 193, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 404, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:13:55:436.141 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011135616512, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:13:55:436.219 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011135616512, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:55:436.272 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011135616512, Pid 113882]: SANDESH: Send FAILED: 1756692835435327 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-3993-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.14 server_addrs= [ [ (*_iter6) = 10.20.0.14:5269, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.14 server_addrs= [ [ (*_iter6) = 10.20.0.14:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.14, XMPP:dns-server:10.20.0.14, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:13:55:436.292 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011135616512, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:13:55:436.314 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011135616512, Pid 113882]: Processing scm::EvTcpDeleteSession in state Idle 2025-09-01 Mon 02:13:55:436.373 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011135616512, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:55:436.794 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011135616512, Pid 113882]: SANDESH: Send FAILED: 1756692835435555 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3993-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Idle successful_connections = 0 pid = 113882 http_port = 8085 start_time = 1756692234402667 collector_name = collector_ip = 10.0.0.56:8086 collector_list= [ [ (*_iter40) = 10.0.0.56:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ComputeCpuStateTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 10 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 54 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 90 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 141 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterControlStatsTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 20 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 30 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 196 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 50 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 194, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 405, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:13:55:436.845 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011135616512, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:14:07:071.675 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011171994240, Pid 113882]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.0.0.56:8085::10.0.0.56:55580 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 255 2025-09-01 Mon 02:14:08:638.795 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:08:638.924 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: SANDESH: Send FAILED: 1756692848638724 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.14 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:08:638.952 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:08:639.063 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011156608512, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:08:639.118 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011171994240, Pid 113882]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.14 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:14:08:639.203 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011156608512, Pid 113882]: SANDESH: Send FAILED: 1756692848639008 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.14 state = Connect event = Connect Event local_port = 44913 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:08:639.342 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011156608512, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:08:639.386 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011160806912, Pid 113882]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:14:08:639.577 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:08:639.617 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: SANDESH: Send FAILED: 1756692848639488 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.14 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:08:639.637 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:08:639.658 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:08:639.716 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: SANDESH: Send FAILED: 1756692848639549 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.14 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:08:639.734 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:20:169.040 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011171994240, Pid 113882]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.0.0.56:8085::10.0.0.56:37152 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 255 2025-09-01 Mon 02:14:22:813.055 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011160806912, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:22:813.209 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011160806912, Pid 113882]: SANDESH: Send FAILED: 1756692862812994 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.14 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:22:813.244 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011160806912, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:22:813.291 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011171994240, Pid 113882]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.14 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:14:22:813.414 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011152410112, Pid 113882]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:14:22:813.477 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011135616512, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:22:813.598 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011135616512, Pid 113882]: SANDESH: Send FAILED: 1756692862813354 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.14 state = Connect event = Connect Event local_port = 40211 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:22:813.660 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011135616512, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:22:813.695 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011135616512, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:22:813.742 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011135616512, Pid 113882]: SANDESH: Send FAILED: 1756692862813449 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.14 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:22:813.769 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011135616512, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:22:813.797 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011135616512, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:22:813.824 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011135616512, Pid 113882]: SANDESH: Send FAILED: 1756692862813460 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.14 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:22:813.842 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011135616512, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:24:415.250 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011156608512, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:24:415.385 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011156608512, Pid 113882]: SANDESH: Send FAILED: 1756692864415065 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3993-1:Compute:contrail-vrouter-agent:0 sm_queue_count = 0 sm_stats= [ ev_stats= [ [ [ event = scm::EvCollectorUpdate enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvIdleHoldTimerExpired enqueues = 24 dequeues = 24 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvSandeshSend enqueues = 416 dequeues = 416 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvStart enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpConnectFail enqueues = 25 dequeues = 25 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpDeleteSession enqueues = 25 dequeues = 25 enqueue_fails = 0 dequeue_fails = 0 ], [ event = enqueues = 492 dequeues = 492 enqueue_fails = 0 dequeue_fails = 0 ], ] ] state = Idle state_since = 1756692835435444 last_state = Idle last_event = scm::EvSandeshSend last_event_at = 1756692862813793 ] max_sm_queue_count = 5 ] 2025-09-01 Mon 02:14:24:415.412 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011156608512, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:14:24:415.445 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011156608512, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:24:415.805 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011156608512, Pid 113882]: SANDESH: Send FAILED: 1756692864415173 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3993-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Idle successful_connections = 0 pid = 113882 http_port = 8085 start_time = 1756692234402667 collector_name = collector_ip = 10.0.0.56:8086 collector_list= [ [ (*_iter40) = 10.0.0.56:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ComputeCpuStateTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 10 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 55 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 92 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 145 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterControlStatsTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 20 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 30 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 204 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 52 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 200, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 416, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:14:24:415.853 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011156608512, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:14:24:422.175 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011152410112, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:24:422.357 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011152410112, Pid 113882]: SANDESH: Send FAILED: 1756692864422134 [SYS_NOTICE]: VrouterStats: data= [ name = cn-jenkins-deploy-platform-ansible-os-3993-1 in_tpkts = 190 in_bytes = 8176 out_tpkts = 193 out_bytes = 8386 raw_vhost_stats= [ name = vhost0 in_pkts = 190 in_bytes = 8176 out_pkts = 193 out_bytes = 8386 speed = -1 duplexity = -1 ] raw_vhost_drop_stats= [ ds_discard = 0 ds_pull = 0 ds_invalid_if = 0 ds_invalid_arp = 0 ds_trap_no_if = 0 ds_nowhere_to_go = 0 ds_flow_queue_limit_exceeded = 0 ds_flow_no_memory = 0 ds_flow_invalid_protocol = 0 ds_flow_nat_no_rflow = 0 ds_flow_action_drop = 0 ds_flow_action_invalid = 0 ds_flow_unusable = 0 ds_flow_table_full = 0 ds_interface_tx_discard = 0 ds_interface_drop = 0 ds_duplicated = 0 ds_push = 0 ds_ttl_exceeded = 0 ds_invalid_nh = 0 ds_invalid_label = 0 ds_invalid_protocol = 0 ds_interface_rx_discard = 0 ds_invalid_mcast_source = 0 ds_head_alloc_fail = 0 ds_pcow_fail = 0 ds_mcast_clone_fail = 0 ds_rewrite_fail = 0 ds_misc = 0 ds_invalid_packet = 0 ds_cksum_err = 0 ds_no_fmd = 0 ds_invalid_vnid = 0 ds_frag_err = 0 ds_invalid_source = 0 ds_mcast_df_bit = 0 ds_l2_no_route = 0 ds_vlan_fwd_tx = 0 ds_vlan_fwd_enq = 0 ds_drop_new_flow = 0 ds_flow_evict = 0 ds_trap_original = 0 ds_no_memory = 0 ds_fragment_queue_fail = 0 ds_no_frag_entry = 0 ds_icmp_error = 0 ds_clone_fail = 0 ds_drop_pkts = 0 ] phy_if_5min_usage= [ [ [ name = ens6 in_bandwidth_usage = 5 out_bandwidth_usage = 35 ], ] ] raw_drop_stats= [ ds_discard = 0 ds_pull = 0 ds_invalid_if = 0 ds_invalid_arp = 0 ds_trap_no_if = 0 ds_nowhere_to_go = 0 ds_flow_queue_limit_exceeded = 0 ds_flow_no_memory = 0 ds_flow_invalid_protocol = 0 ds_flow_nat_no_rflow = 0 ds_flow_action_drop = 0 ds_flow_action_invalid = 0 ds_flow_unusable = 0 ds_flow_table_full = 0 ds_interface_tx_discard = 0 ds_interface_drop = 0 ds_duplicated = 0 ds_push = 0 ds_ttl_exceeded = 0 ds_invalid_nh = 0 ds_invalid_label = 0 ds_invalid_protocol = 0 ds_interface_rx_discard = 0 ds_invalid_mcast_source = 0 ds_head_alloc_fail = 0 ds_pcow_fail = 0 ds_mcast_clone_fail = 0 ds_rewrite_fail = 0 ds_misc = 0 ds_invalid_packet = 0 ds_cksum_err = 0 ds_no_fmd = 0 ds_invalid_vnid = 0 ds_frag_err = 0 ds_invalid_source = 0 ds_mcast_df_bit = 0 ds_l2_no_route = 0 ds_vlan_fwd_tx = 0 ds_vlan_fwd_enq = 0 ds_drop_new_flow = 0 ds_flow_evict = 0 ds_trap_original = 0 ds_no_memory = 0 ds_fragment_queue_fail = 0 ds_no_frag_entry = 0 ds_icmp_error = 0 ds_clone_fail = 0 ds_drop_pkts = 0 ] flow_rate= [ added_flows = 0 max_flow_adds_per_second = 0 min_flow_adds_per_second = 0 deleted_flows = 0 max_flow_deletes_per_second = 0 min_flow_deletes_per_second = 0 active_flows = 0 hold_flows = 0 ] phy_band_in_bps= [ [ _iter260->first = ens6 _iter260->second = 0, ] ] phy_band_out_bps= [ [ _iter261->first = ens6 _iter261->second = 33, ] ] raw_phy_if_stats= [ [ _iter262->first = ens6 [ in_pkts = 10 in_bytes = 700 out_pkts = 190 out_bytes = 8176 ], ] ] raw_phy_if_drop_stats= [ [ _iter263->first = ens6 [ ds_discard = 0 ds_pull = 0 ds_invalid_if = 0 ds_invalid_arp = 0 ds_trap_no_if = 0 ds_nowhere_to_go = 0 ds_flow_queue_limit_exceeded = 0 ds_flow_no_memory = 0 ds_flow_invalid_protocol = 0 ds_flow_nat_no_rflow = 0 ds_flow_action_drop = 0 ds_flow_action_invalid = 0 ds_flow_unusable = 0 ds_flow_table_full = 0 ds_interface_tx_discard = 0 ds_interface_drop = 0 ds_duplicated = 0 ds_push = 0 ds_ttl_exceeded = 0 ds_invalid_nh = 0 ds_invalid_label = 0 ds_invalid_protocol = 0 ds_interface_rx_discard = 0 ds_invalid_mcast_source = 0 ds_head_alloc_fail = 0 ds_pcow_fail = 0 ds_mcast_clone_fail = 0 ds_rewrite_fail = 0 ds_misc = 0 ds_invalid_packet = 0 ds_cksum_err = 0 ds_no_fmd = 0 ds_invalid_vnid = 0 ds_frag_err = 0 ds_invalid_source = 0 ds_mcast_df_bit = 0 ds_l2_no_route = 0 ds_vlan_fwd_tx = 0 ds_vlan_fwd_enq = 0 ds_drop_new_flow = 0 ds_flow_evict = 0 ds_trap_original = 0 ds_no_memory = 0 ds_fragment_queue_fail = 0 ds_no_frag_entry = 0 ds_icmp_error = 0 ds_clone_fail = 0 ds_drop_pkts = 0 ], ] ] ] 2025-09-01 Mon 02:14:24:422.387 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011152410112, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: VrouterStats 2025-09-01 Mon 02:14:24:422.398 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011152410112, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:24:422.426 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011152410112, Pid 113882]: SANDESH: Send FAILED: 1756692864422271 [SYS_NOTICE]: VrouterControlStatsTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3993-1 raw_xmpp_stats= [ [ _iter306->first = 10.20.0.14 [ reconnects = 0 in_msgs = 0 out_msgs = 0 ], ] ] raw_ifmap_stats= [ [ _iter307->first = link_delete_parse_errors _iter307->second = 0, _iter307->first = link_update_parse_errors _iter307->second = 0, _iter307->first = node_delete_parse_errors _iter307->second = 0, _iter307->first = node_update_parse_errors _iter307->second = 0, ] ] raw_rt_table_size= [ [ _iter308->first = default-domain:default-project:ip-fabric:__default__ [ inet4_unicast = 4 inet4_multicast = 0 evpn = 0 bridge = 2 inet6_unicast = 2 ], _iter308->first = default-domain:default-project:ip-fabric:ip-fabric [ inet4_unicast = 2 inet4_multicast = 0 evpn = 1 bridge = 2 inet6_unicast = 0 ], ] ] ] 2025-09-01 Mon 02:14:24:422.433 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011152410112, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: VrouterControlStatsTrace 2025-09-01 Mon 02:14:25:435.754 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011152410112, Pid 113882]: Processing scm::EvIdleHoldTimerExpired in state Idle 2025-09-01 Mon 02:14:25:436.316 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011152410112, Pid 113882]: Connect : Start Connect timer 10.0.0.56:8086 2025-09-01 Mon 02:14:25:436.378 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011171994240, Pid 113882]: -(-1) OnSessionEvent TCP Connect Failed 2025-09-01 Mon 02:14:25:436.497 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011171994240, Pid 113882]: -(-1):EvTcpConnectFail 2025-09-01 Mon 02:14:25:436.620 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011152410112, Pid 113882]: Processing scm::EvSandeshSend in state Connect 2025-09-01 Mon 02:14:25:436.692 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011152410112, Pid 113882]: SANDESH: Send FAILED: 1756692865435957 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-3993-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.14 server_addrs= [ [ (*_iter6) = 10.20.0.14:5269, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.14 server_addrs= [ [ (*_iter6) = 10.20.0.14:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.56:8086, ] ] status = Initializing description = Idle : EvIdleHoldTimerExpired -> Connect ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.14, XMPP:dns-server:10.20.0.14, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:14:25:436.716 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011152410112, Pid 113882]: Wrong state: Connect for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:14:25:436.742 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011152410112, Pid 113882]: Processing scm::EvTcpConnectFail in state Connect 2025-09-01 Mon 02:14:25:436.786 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011152410112, Pid 113882]: Connect : EvTcpConnectFail 2025-09-01 Mon 02:14:25:436.894 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011152410112, Pid 113882]: TCP [SYS_ERR]: TcpSessionMessageLog: Session -(-1) > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:14:25:436.962 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011152410112, Pid 113882]: Idle 2025-09-01 Mon 02:14:25:437.095 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011152410112, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:25:437.514 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011152410112, Pid 113882]: SANDESH: Send FAILED: 1756692865436576 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3993-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Connect successful_connections = 0 pid = 113882 http_port = 8085 start_time = 1756692234402667 collector_name = collector_ip = 10.0.0.56:8086 collector_list= [ [ (*_iter40) = 10.0.0.56:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ComputeCpuStateTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 10 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 55 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 94 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 145 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterControlStatsTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 21 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 31 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 204 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 52 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 200, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 420, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:14:25:437.569 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011152410112, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:14:25:437.600 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011152410112, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:25:437.649 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011152410112, Pid 113882]: SANDESH: Send FAILED: 1756692865436841 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-3993-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.14 server_addrs= [ [ (*_iter6) = 10.20.0.14:5269, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.14 server_addrs= [ [ (*_iter6) = 10.20.0.14:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.14, XMPP:dns-server:10.20.0.14, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:14:25:437.668 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011152410112, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:14:25:437.690 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011152410112, Pid 113882]: Processing scm::EvTcpDeleteSession in state Idle 2025-09-01 Mon 02:14:25:437.754 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011152410112, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:25:438.133 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011152410112, Pid 113882]: SANDESH: Send FAILED: 1756692865437070 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3993-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Idle successful_connections = 0 pid = 113882 http_port = 8085 start_time = 1756692234402667 collector_name = collector_ip = 10.0.0.56:8086 collector_list= [ [ (*_iter40) = 10.0.0.56:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ComputeCpuStateTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 10 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 56 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 94 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 146 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterControlStatsTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 21 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 31 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 204 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 52 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 201, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 421, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:14:25:438.202 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011152410112, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:14:33:257.271 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011171994240, Pid 113882]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.0.0.56:8085::10.0.0.56:59290 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 255 2025-09-01 Mon 02:14:38:069.885 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011152410112, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:38:069.972 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011152410112, Pid 113882]: SANDESH: Send FAILED: 1756692878069834 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.14 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:38:069.991 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011152410112, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:38:070.258 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011160806912, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:38:070.366 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011160806912, Pid 113882]: SANDESH: Send FAILED: 1756692878070170 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.14 state = Connect event = Connect Event local_port = 39767 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:38:070.429 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011160806912, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:38:070.270 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011171994240, Pid 113882]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.14 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:14:38:070.665 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011160806912, Pid 113882]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:14:38:070.774 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:38:070.862 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: SANDESH: Send FAILED: 1756692878070727 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.14 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:38:070.888 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:38:070.913 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:38:070.942 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: SANDESH: Send FAILED: 1756692878070738 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.14 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:38:070.960 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:40:386.108 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011171994240, Pid 113882]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.20.0.14:8085::10.20.0.14:48720 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 255 2025-09-01 Mon 02:14:40:409.411 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011171994240, Pid 113882]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.20.0.14:8085::10.20.0.14:48736 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 255 2025-09-01 Mon 02:14:46:076.934 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011171994240, Pid 113882]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.0.0.56:8085::10.0.0.56:41864 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 255 2025-09-01 Mon 02:14:54:415.535 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011152410112, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:54:415.636 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011152410112, Pid 113882]: SANDESH: Send FAILED: 1756692894415371 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3993-1:Compute:contrail-vrouter-agent:0 sm_queue_count = 0 sm_stats= [ ev_stats= [ [ [ event = scm::EvCollectorUpdate enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvIdleHoldTimerExpired enqueues = 25 dequeues = 25 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvSandeshSend enqueues = 428 dequeues = 428 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvStart enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpConnectFail enqueues = 26 dequeues = 26 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpDeleteSession enqueues = 26 dequeues = 26 enqueue_fails = 0 dequeue_fails = 0 ], [ event = enqueues = 507 dequeues = 507 enqueue_fails = 0 dequeue_fails = 0 ], ] ] state = Idle state_since = 1756692865436959 last_state = Idle last_event = scm::EvSandeshSend last_event_at = 1756692878070910 ] max_sm_queue_count = 5 ] 2025-09-01 Mon 02:14:54:415.651 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011152410112, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:14:54:415.661 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011152410112, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:54:415.855 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011152410112, Pid 113882]: SANDESH: Send FAILED: 1756692894415494 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3993-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Idle successful_connections = 0 pid = 113882 http_port = 8085 start_time = 1756692234402667 collector_name = collector_ip = 10.0.0.56:8086 collector_list= [ [ (*_iter40) = 10.0.0.56:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ComputeCpuStateTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 10 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 57 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 96 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 151 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterControlStatsTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 21 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 31 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 208 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 53 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 207, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 428, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:14:54:415.884 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011152410112, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:14:54:422.655 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011152410112, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:54:422.736 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011152410112, Pid 113882]: SANDESH: Send FAILED: 1756692894422542 [SYS_NOTICE]: VrouterStats: data= [ name = cn-jenkins-deploy-platform-ansible-os-3993-1 cpu_info= [ num_cpu = 8 sys_mem_info= [ total = 32 used = 0 free = 0 buffers = 0 cached = 3882065592 node_type = ] meminfo= [ virt = 32598 peakvirt = 0 res = 0 ] cpuload= [ one_min_avg = 0.18125 five_min_avg = 0.1925 fifteen_min_avg = 0.21875 ] cpu_share = 0.4 ] ] 2025-09-01 Mon 02:14:54:422.748 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011152410112, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: VrouterStats 2025-09-01 Mon 02:14:54:422.770 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011152410112, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:54:422.791 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011152410112, Pid 113882]: SANDESH: Send FAILED: 1756692894422616 [SYS_NOTICE]: ComputeCpuStateTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3993-1 cpu_info= [ [ [ mem_virt = 32598 cpu_share = 0.4 used_sys_mem = 412901704 one_min_cpuload = 0.18125 mem_res = 0 ], ] ] ] 2025-09-01 Mon 02:14:54:422.800 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011152410112, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: ComputeCpuStateTrace 2025-09-01 Mon 02:14:54:422.808 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011152410112, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:54:422.961 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011152410112, Pid 113882]: SANDESH: Send FAILED: 1756692894422668 [SYS_NOTICE]: VrouterStats: data= [ name = cn-jenkins-deploy-platform-ansible-os-3993-1 in_tpkts = 193 in_bytes = 8302 out_tpkts = 196 out_bytes = 8512 raw_vhost_stats= [ name = vhost0 in_pkts = 193 in_bytes = 8302 out_pkts = 196 out_bytes = 8512 speed = -1 duplexity = -1 ] raw_vhost_drop_stats= [ ds_discard = 0 ds_pull = 0 ds_invalid_if = 0 ds_invalid_arp = 0 ds_trap_no_if = 0 ds_nowhere_to_go = 0 ds_flow_queue_limit_exceeded = 0 ds_flow_no_memory = 0 ds_flow_invalid_protocol = 0 ds_flow_nat_no_rflow = 0 ds_flow_action_drop = 0 ds_flow_action_invalid = 0 ds_flow_unusable = 0 ds_flow_table_full = 0 ds_interface_tx_discard = 0 ds_interface_drop = 0 ds_duplicated = 0 ds_push = 0 ds_ttl_exceeded = 0 ds_invalid_nh = 0 ds_invalid_label = 0 ds_invalid_protocol = 0 ds_interface_rx_discard = 0 ds_invalid_mcast_source = 0 ds_head_alloc_fail = 0 ds_pcow_fail = 0 ds_mcast_clone_fail = 0 ds_rewrite_fail = 0 ds_misc = 0 ds_invalid_packet = 0 ds_cksum_err = 0 ds_no_fmd = 0 ds_invalid_vnid = 0 ds_frag_err = 0 ds_invalid_source = 0 ds_mcast_df_bit = 0 ds_l2_no_route = 0 ds_vlan_fwd_tx = 0 ds_vlan_fwd_enq = 0 ds_drop_new_flow = 0 ds_flow_evict = 0 ds_trap_original = 0 ds_no_memory = 0 ds_fragment_queue_fail = 0 ds_no_frag_entry = 0 ds_icmp_error = 0 ds_clone_fail = 0 ds_drop_pkts = 0 ] raw_drop_stats= [ ds_discard = 0 ds_pull = 0 ds_invalid_if = 0 ds_invalid_arp = 0 ds_trap_no_if = 0 ds_nowhere_to_go = 0 ds_flow_queue_limit_exceeded = 0 ds_flow_no_memory = 0 ds_flow_invalid_protocol = 0 ds_flow_nat_no_rflow = 0 ds_flow_action_drop = 0 ds_flow_action_invalid = 0 ds_flow_unusable = 0 ds_flow_table_full = 0 ds_interface_tx_discard = 0 ds_interface_drop = 0 ds_duplicated = 0 ds_push = 0 ds_ttl_exceeded = 0 ds_invalid_nh = 0 ds_invalid_label = 0 ds_invalid_protocol = 0 ds_interface_rx_discard = 0 ds_invalid_mcast_source = 0 ds_head_alloc_fail = 0 ds_pcow_fail = 0 ds_mcast_clone_fail = 0 ds_rewrite_fail = 0 ds_misc = 0 ds_invalid_packet = 0 ds_cksum_err = 0 ds_no_fmd = 0 ds_invalid_vnid = 0 ds_frag_err = 0 ds_invalid_source = 0 ds_mcast_df_bit = 0 ds_l2_no_route = 0 ds_vlan_fwd_tx = 0 ds_vlan_fwd_enq = 0 ds_drop_new_flow = 0 ds_flow_evict = 0 ds_trap_original = 0 ds_no_memory = 0 ds_fragment_queue_fail = 0 ds_no_frag_entry = 0 ds_icmp_error = 0 ds_clone_fail = 0 ds_drop_pkts = 0 ] flow_rate= [ added_flows = 0 max_flow_adds_per_second = 0 min_flow_adds_per_second = 0 deleted_flows = 0 max_flow_deletes_per_second = 0 min_flow_deletes_per_second = 0 active_flows = 0 hold_flows = 0 ] raw_phy_if_stats= [ [ _iter262->first = ens6 [ in_pkts = 10 in_bytes = 700 out_pkts = 193 out_bytes = 8302 ], ] ] raw_phy_if_drop_stats= [ [ _iter263->first = ens6 [ ds_discard = 0 ds_pull = 0 ds_invalid_if = 0 ds_invalid_arp = 0 ds_trap_no_if = 0 ds_nowhere_to_go = 0 ds_flow_queue_limit_exceeded = 0 ds_flow_no_memory = 0 ds_flow_invalid_protocol = 0 ds_flow_nat_no_rflow = 0 ds_flow_action_drop = 0 ds_flow_action_invalid = 0 ds_flow_unusable = 0 ds_flow_table_full = 0 ds_interface_tx_discard = 0 ds_interface_drop = 0 ds_duplicated = 0 ds_push = 0 ds_ttl_exceeded = 0 ds_invalid_nh = 0 ds_invalid_label = 0 ds_invalid_protocol = 0 ds_interface_rx_discard = 0 ds_invalid_mcast_source = 0 ds_head_alloc_fail = 0 ds_pcow_fail = 0 ds_mcast_clone_fail = 0 ds_rewrite_fail = 0 ds_misc = 0 ds_invalid_packet = 0 ds_cksum_err = 0 ds_no_fmd = 0 ds_invalid_vnid = 0 ds_frag_err = 0 ds_invalid_source = 0 ds_mcast_df_bit = 0 ds_l2_no_route = 0 ds_vlan_fwd_tx = 0 ds_vlan_fwd_enq = 0 ds_drop_new_flow = 0 ds_flow_evict = 0 ds_trap_original = 0 ds_no_memory = 0 ds_fragment_queue_fail = 0 ds_no_frag_entry = 0 ds_icmp_error = 0 ds_clone_fail = 0 ds_drop_pkts = 0 ], ] ] ] 2025-09-01 Mon 02:14:54:422.977 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011152410112, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: VrouterStats 2025-09-01 Mon 02:14:54:422.986 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011152410112, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:54:423.017 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011152410112, Pid 113882]: SANDESH: Send FAILED: 1756692894422746 [SYS_NOTICE]: VrouterControlStatsTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3993-1 raw_xmpp_stats= [ [ _iter306->first = 10.20.0.14 [ reconnects = 0 in_msgs = 0 out_msgs = 0 ], ] ] raw_ifmap_stats= [ [ _iter307->first = link_delete_parse_errors _iter307->second = 0, _iter307->first = link_update_parse_errors _iter307->second = 0, _iter307->first = node_delete_parse_errors _iter307->second = 0, _iter307->first = node_update_parse_errors _iter307->second = 0, ] ] raw_rt_table_size= [ [ _iter308->first = default-domain:default-project:ip-fabric:__default__ [ inet4_unicast = 4 inet4_multicast = 0 evpn = 0 bridge = 2 inet6_unicast = 2 ], _iter308->first = default-domain:default-project:ip-fabric:ip-fabric [ inet4_unicast = 2 inet4_multicast = 0 evpn = 1 bridge = 2 inet6_unicast = 0 ], ] ] ] 2025-09-01 Mon 02:14:54:423.025 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011152410112, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: VrouterControlStatsTrace 2025-09-01 Mon 02:14:54:673.878 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011135616512, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:54:674.001 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011135616512, Pid 113882]: SANDESH: Send FAILED: 1756692894673807 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.14 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:54:674.029 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011135616512, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:54:674.271 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011160806912, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:54:674.272 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011171994240, Pid 113882]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.14 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:14:54:674.393 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011160806912, Pid 113882]: SANDESH: Send FAILED: 1756692894674154 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.14 state = Connect event = Connect Event local_port = 43789 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:54:674.515 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011160806912, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:54:674.571 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011139814912, Pid 113882]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:14:54:674.810 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011160806912, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:54:674.860 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011160806912, Pid 113882]: SANDESH: Send FAILED: 1756692894674698 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.14 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:54:674.881 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011160806912, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:54:674.903 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011160806912, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:54:674.931 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011160806912, Pid 113882]: SANDESH: Send FAILED: 1756692894674753 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.14 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:54:674.948 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011160806912, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:55:437.228 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011152410112, Pid 113882]: Processing scm::EvIdleHoldTimerExpired in state Idle 2025-09-01 Mon 02:14:55:437.500 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011152410112, Pid 113882]: Connect : Start Connect timer 10.0.0.56:8086 2025-09-01 Mon 02:14:55:437.561 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011171994240, Pid 113882]: -(-1) OnSessionEvent TCP Connect Failed 2025-09-01 Mon 02:14:55:437.628 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011152410112, Pid 113882]: Processing scm::EvSandeshSend in state Connect 2025-09-01 Mon 02:14:55:437.632 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011171994240, Pid 113882]: -(-1):EvTcpConnectFail 2025-09-01 Mon 02:14:55:437.674 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011152410112, Pid 113882]: SANDESH: Send FAILED: 1756692895437318 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-3993-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.14 server_addrs= [ [ (*_iter6) = 10.20.0.14:5269, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.14 server_addrs= [ [ (*_iter6) = 10.20.0.14:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.56:8086, ] ] status = Initializing description = Idle : EvIdleHoldTimerExpired -> Connect ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.14, XMPP:dns-server:10.20.0.14, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:14:55:437.688 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011152410112, Pid 113882]: Wrong state: Connect for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:14:55:437.701 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011152410112, Pid 113882]: Processing scm::EvSandeshSend in state Connect 2025-09-01 Mon 02:14:55:437.927 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011152410112, Pid 113882]: SANDESH: Send FAILED: 1756692895437606 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3993-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Connect successful_connections = 0 pid = 113882 http_port = 8085 start_time = 1756692234402667 collector_name = collector_ip = 10.0.0.56:8086 collector_list= [ [ (*_iter40) = 10.0.0.56:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ComputeCpuStateTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 11 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 57 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 98 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 152 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterControlStatsTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 22 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 33 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 212 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 54 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 209, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 438, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:14:55:437.959 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011152410112, Pid 113882]: Wrong state: Connect for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:14:55:437.974 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011152410112, Pid 113882]: Processing scm::EvTcpConnectFail in state Connect 2025-09-01 Mon 02:14:55:437.984 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011152410112, Pid 113882]: Connect : EvTcpConnectFail 2025-09-01 Mon 02:14:55:438.039 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011152410112, Pid 113882]: TCP [SYS_ERR]: TcpSessionMessageLog: Session -(-1) > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:14:55:438.073 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011152410112, Pid 113882]: Idle 2025-09-01 Mon 02:14:55:438.143 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011152410112, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:55:438.170 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011152410112, Pid 113882]: SANDESH: Send FAILED: 1756692895438011 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-3993-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.14 server_addrs= [ [ (*_iter6) = 10.20.0.14:5269, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.14 server_addrs= [ [ (*_iter6) = 10.20.0.14:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.14, XMPP:dns-server:10.20.0.14, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:14:55:438.179 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011152410112, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:14:55:438.202 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011152410112, Pid 113882]: Processing scm::EvTcpDeleteSession in state Idle 2025-09-01 Mon 02:14:55:438.236 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011152410112, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:55:438.445 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011152410112, Pid 113882]: SANDESH: Send FAILED: 1756692895438129 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3993-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Idle successful_connections = 0 pid = 113882 http_port = 8085 start_time = 1756692234402667 collector_name = collector_ip = 10.0.0.56:8086 collector_list= [ [ (*_iter40) = 10.0.0.56:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ComputeCpuStateTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 11 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 58 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 99 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 153 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterControlStatsTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 22 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 33 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 212 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 54 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 210, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 440, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:14:55:438.470 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011152410112, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:14:59:714.737 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011171994240, Pid 113882]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.0.0.56:8085::10.0.0.56:33484 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 255 2025-09-01 Mon 02:15:06:421.006 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011148211712, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:06:421.163 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011148211712, Pid 113882]: SANDESH: Send FAILED: 1756692906420948 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.14 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:06:421.226 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011148211712, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:06:421.248 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011171994240, Pid 113882]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.14 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:15:06:421.266 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011148211712, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:06:421.415 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011148211712, Pid 113882]: SANDESH: Send FAILED: 1756692906421129 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.14 state = Connect event = Connect Event local_port = 43971 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:06:421.446 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011148211712, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:06:421.420 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011152410112, Pid 113882]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:15:06:421.685 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011156608512, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:06:421.779 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011156608512, Pid 113882]: SANDESH: Send FAILED: 1756692906421565 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.14 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:06:421.805 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011156608512, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:06:421.831 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011156608512, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:06:421.860 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011156608512, Pid 113882]: SANDESH: Send FAILED: 1756692906421611 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.14 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:06:421.878 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011156608512, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:12:783.819 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011171994240, Pid 113882]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.0.0.56:8085::10.0.0.56:40418 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 255 2025-09-01 Mon 02:15:24:105.171 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011152410112, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:24:105.266 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011152410112, Pid 113882]: SANDESH: Send FAILED: 1756692924105120 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.14 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:24:105.293 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011152410112, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:24:105.548 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011148211712, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:24:105.591 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011171994240, Pid 113882]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.14 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:15:24:105.660 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011148211712, Pid 113882]: SANDESH: Send FAILED: 1756692924105479 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.14 state = Connect event = Connect Event local_port = 46881 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:24:105.800 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011148211712, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:24:105.922 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011160806912, Pid 113882]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:15:24:106.118 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:24:106.146 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: SANDESH: Send FAILED: 1756692924106045 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.14 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:24:106.157 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:24:106.168 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:24:106.199 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: SANDESH: Send FAILED: 1756692924106102 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.14 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:24:106.210 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:24:416.109 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011135616512, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:24:416.307 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011135616512, Pid 113882]: SANDESH: Send FAILED: 1756692924415809 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3993-1:Compute:contrail-vrouter-agent:0 sm_queue_count = 0 sm_stats= [ ev_stats= [ [ [ event = scm::EvCollectorUpdate enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvIdleHoldTimerExpired enqueues = 26 dequeues = 26 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvSandeshSend enqueues = 450 dequeues = 450 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvStart enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpConnectFail enqueues = 27 dequeues = 27 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpDeleteSession enqueues = 27 dequeues = 27 enqueue_fails = 0 dequeue_fails = 0 ], [ event = enqueues = 532 dequeues = 532 enqueue_fails = 0 dequeue_fails = 0 ], ] ] state = Idle state_since = 1756692895438072 last_state = Idle last_event = scm::EvSandeshSend last_event_at = 1756692924106166 ] max_sm_queue_count = 5 ] 2025-09-01 Mon 02:15:24:416.367 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011135616512, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:15:24:416.393 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011135616512, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:24:416.977 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011135616512, Pid 113882]: SANDESH: Send FAILED: 1756692924416009 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3993-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Idle successful_connections = 0 pid = 113882 http_port = 8085 start_time = 1756692234402667 collector_name = collector_ip = 10.0.0.56:8086 collector_list= [ [ (*_iter40) = 10.0.0.56:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ComputeCpuStateTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 11 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 59 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 100 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 157 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterControlStatsTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 22 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 33 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 220 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 56 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 216, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 450, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:15:24:417.043 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011135616512, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:15:24:423.118 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011135616512, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:24:423.548 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011135616512, Pid 113882]: SANDESH: Send FAILED: 1756692924423008 [SYS_NOTICE]: VrouterStats: data= [ name = cn-jenkins-deploy-platform-ansible-os-3993-1 in_tpkts = 196 in_bytes = 8428 out_tpkts = 199 out_bytes = 8638 raw_vhost_stats= [ name = vhost0 in_pkts = 196 in_bytes = 8428 out_pkts = 199 out_bytes = 8638 speed = -1 duplexity = -1 ] raw_vhost_drop_stats= [ ds_discard = 0 ds_pull = 0 ds_invalid_if = 0 ds_invalid_arp = 0 ds_trap_no_if = 0 ds_nowhere_to_go = 0 ds_flow_queue_limit_exceeded = 0 ds_flow_no_memory = 0 ds_flow_invalid_protocol = 0 ds_flow_nat_no_rflow = 0 ds_flow_action_drop = 0 ds_flow_action_invalid = 0 ds_flow_unusable = 0 ds_flow_table_full = 0 ds_interface_tx_discard = 0 ds_interface_drop = 0 ds_duplicated = 0 ds_push = 0 ds_ttl_exceeded = 0 ds_invalid_nh = 0 ds_invalid_label = 0 ds_invalid_protocol = 0 ds_interface_rx_discard = 0 ds_invalid_mcast_source = 0 ds_head_alloc_fail = 0 ds_pcow_fail = 0 ds_mcast_clone_fail = 0 ds_rewrite_fail = 0 ds_misc = 0 ds_invalid_packet = 0 ds_cksum_err = 0 ds_no_fmd = 0 ds_invalid_vnid = 0 ds_frag_err = 0 ds_invalid_source = 0 ds_mcast_df_bit = 0 ds_l2_no_route = 0 ds_vlan_fwd_tx = 0 ds_vlan_fwd_enq = 0 ds_drop_new_flow = 0 ds_flow_evict = 0 ds_trap_original = 0 ds_no_memory = 0 ds_fragment_queue_fail = 0 ds_no_frag_entry = 0 ds_icmp_error = 0 ds_clone_fail = 0 ds_drop_pkts = 0 ] raw_drop_stats= [ ds_discard = 0 ds_pull = 0 ds_invalid_if = 0 ds_invalid_arp = 0 ds_trap_no_if = 0 ds_nowhere_to_go = 0 ds_flow_queue_limit_exceeded = 0 ds_flow_no_memory = 0 ds_flow_invalid_protocol = 0 ds_flow_nat_no_rflow = 0 ds_flow_action_drop = 0 ds_flow_action_invalid = 0 ds_flow_unusable = 0 ds_flow_table_full = 0 ds_interface_tx_discard = 0 ds_interface_drop = 0 ds_duplicated = 0 ds_push = 0 ds_ttl_exceeded = 0 ds_invalid_nh = 0 ds_invalid_label = 0 ds_invalid_protocol = 0 ds_interface_rx_discard = 0 ds_invalid_mcast_source = 0 ds_head_alloc_fail = 0 ds_pcow_fail = 0 ds_mcast_clone_fail = 0 ds_rewrite_fail = 0 ds_misc = 0 ds_invalid_packet = 0 ds_cksum_err = 0 ds_no_fmd = 0 ds_invalid_vnid = 0 ds_frag_err = 0 ds_invalid_source = 0 ds_mcast_df_bit = 0 ds_l2_no_route = 0 ds_vlan_fwd_tx = 0 ds_vlan_fwd_enq = 0 ds_drop_new_flow = 0 ds_flow_evict = 0 ds_trap_original = 0 ds_no_memory = 0 ds_fragment_queue_fail = 0 ds_no_frag_entry = 0 ds_icmp_error = 0 ds_clone_fail = 0 ds_drop_pkts = 0 ] flow_rate= [ added_flows = 0 max_flow_adds_per_second = 0 min_flow_adds_per_second = 0 deleted_flows = 0 max_flow_deletes_per_second = 0 min_flow_deletes_per_second = 0 active_flows = 0 hold_flows = 0 ] phy_band_in_bps= [ [ _iter260->first = ens6 _iter260->second = 0, ] ] phy_band_out_bps= [ [ _iter261->first = ens6 _iter261->second = 33, ] ] raw_phy_if_stats= [ [ _iter262->first = ens6 [ in_pkts = 10 in_bytes = 700 out_pkts = 196 out_bytes = 8428 ], ] ] raw_phy_if_drop_stats= [ [ _iter263->first = ens6 [ ds_discard = 0 ds_pull = 0 ds_invalid_if = 0 ds_invalid_arp = 0 ds_trap_no_if = 0 ds_nowhere_to_go = 0 ds_flow_queue_limit_exceeded = 0 ds_flow_no_memory = 0 ds_flow_invalid_protocol = 0 ds_flow_nat_no_rflow = 0 ds_flow_action_drop = 0 ds_flow_action_invalid = 0 ds_flow_unusable = 0 ds_flow_table_full = 0 ds_interface_tx_discard = 0 ds_interface_drop = 0 ds_duplicated = 0 ds_push = 0 ds_ttl_exceeded = 0 ds_invalid_nh = 0 ds_invalid_label = 0 ds_invalid_protocol = 0 ds_interface_rx_discard = 0 ds_invalid_mcast_source = 0 ds_head_alloc_fail = 0 ds_pcow_fail = 0 ds_mcast_clone_fail = 0 ds_rewrite_fail = 0 ds_misc = 0 ds_invalid_packet = 0 ds_cksum_err = 0 ds_no_fmd = 0 ds_invalid_vnid = 0 ds_frag_err = 0 ds_invalid_source = 0 ds_mcast_df_bit = 0 ds_l2_no_route = 0 ds_vlan_fwd_tx = 0 ds_vlan_fwd_enq = 0 ds_drop_new_flow = 0 ds_flow_evict = 0 ds_trap_original = 0 ds_no_memory = 0 ds_fragment_queue_fail = 0 ds_no_frag_entry = 0 ds_icmp_error = 0 ds_clone_fail = 0 ds_drop_pkts = 0 ], ] ] ] 2025-09-01 Mon 02:15:24:423.601 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011135616512, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: VrouterStats 2025-09-01 Mon 02:15:24:423.634 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011135616512, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:24:423.709 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011135616512, Pid 113882]: SANDESH: Send FAILED: 1756692924423111 [SYS_NOTICE]: VrouterControlStatsTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3993-1 raw_xmpp_stats= [ [ _iter306->first = 10.20.0.14 [ reconnects = 0 in_msgs = 0 out_msgs = 0 ], ] ] raw_ifmap_stats= [ [ _iter307->first = link_delete_parse_errors _iter307->second = 0, _iter307->first = link_update_parse_errors _iter307->second = 0, _iter307->first = node_delete_parse_errors _iter307->second = 0, _iter307->first = node_update_parse_errors _iter307->second = 0, ] ] raw_rt_table_size= [ [ _iter308->first = default-domain:default-project:ip-fabric:__default__ [ inet4_unicast = 4 inet4_multicast = 0 evpn = 0 bridge = 2 inet6_unicast = 2 ], _iter308->first = default-domain:default-project:ip-fabric:ip-fabric [ inet4_unicast = 2 inet4_multicast = 0 evpn = 1 bridge = 2 inet6_unicast = 0 ], ] ] ] 2025-09-01 Mon 02:15:24:423.730 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011135616512, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: VrouterControlStatsTrace 2025-09-01 Mon 02:15:25:438.224 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011156608512, Pid 113882]: Processing scm::EvIdleHoldTimerExpired in state Idle 2025-09-01 Mon 02:15:25:438.495 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011156608512, Pid 113882]: Connect : Start Connect timer 10.0.0.56:8086 2025-09-01 Mon 02:15:25:438.571 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011171994240, Pid 113882]: -(-1) OnSessionEvent TCP Connect Failed 2025-09-01 Mon 02:15:25:438.613 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011156608512, Pid 113882]: Processing scm::EvSandeshSend in state Connect 2025-09-01 Mon 02:15:25:438.668 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011171994240, Pid 113882]: -(-1):EvTcpConnectFail 2025-09-01 Mon 02:15:25:438.700 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011156608512, Pid 113882]: SANDESH: Send FAILED: 1756692925438315 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-3993-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.14 server_addrs= [ [ (*_iter6) = 10.20.0.14:5269, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.14 server_addrs= [ [ (*_iter6) = 10.20.0.14:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.56:8086, ] ] status = Initializing description = Idle : EvIdleHoldTimerExpired -> Connect ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.14, XMPP:dns-server:10.20.0.14, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:15:25:438.777 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011156608512, Pid 113882]: Wrong state: Connect for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:15:25:438.788 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011156608512, Pid 113882]: Processing scm::EvSandeshSend in state Connect 2025-09-01 Mon 02:15:25:438.945 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011156608512, Pid 113882]: SANDESH: Send FAILED: 1756692925438592 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3993-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Connect successful_connections = 0 pid = 113882 http_port = 8085 start_time = 1756692234402667 collector_name = collector_ip = 10.0.0.56:8086 collector_list= [ [ (*_iter40) = 10.0.0.56:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ComputeCpuStateTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 11 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 59 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 102 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 157 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterControlStatsTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 23 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 34 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 220 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 56 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 216, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 454, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:15:25:438.975 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011156608512, Pid 113882]: Wrong state: Connect for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:15:25:438.986 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011156608512, Pid 113882]: Processing scm::EvTcpConnectFail in state Connect 2025-09-01 Mon 02:15:25:439.000 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011156608512, Pid 113882]: Connect : EvTcpConnectFail 2025-09-01 Mon 02:15:25:439.050 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011156608512, Pid 113882]: TCP [SYS_ERR]: TcpSessionMessageLog: Session -(-1) > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:15:25:439.082 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011156608512, Pid 113882]: Idle 2025-09-01 Mon 02:15:25:439.143 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011156608512, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:25:439.164 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011156608512, Pid 113882]: SANDESH: Send FAILED: 1756692925439026 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-3993-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.14 server_addrs= [ [ (*_iter6) = 10.20.0.14:5269, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.14 server_addrs= [ [ (*_iter6) = 10.20.0.14:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.14, XMPP:dns-server:10.20.0.14, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:15:25:439.178 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011156608512, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:15:25:439.204 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011156608512, Pid 113882]: Processing scm::EvTcpDeleteSession in state Idle 2025-09-01 Mon 02:15:25:439.234 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011156608512, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:25:439.377 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011156608512, Pid 113882]: SANDESH: Send FAILED: 1756692925439131 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3993-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Idle successful_connections = 0 pid = 113882 http_port = 8085 start_time = 1756692234402667 collector_name = collector_ip = 10.0.0.56:8086 collector_list= [ [ (*_iter40) = 10.0.0.56:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ComputeCpuStateTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 11 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 60 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 103 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 158 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterControlStatsTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 23 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 34 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 220 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 56 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 217, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 456, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:15:25:439.396 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011156608512, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:15:25:691.811 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011171994240, Pid 113882]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.0.0.56:8085::10.0.0.56:50998 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 255 2025-09-01 Mon 02:15:34:771.990 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011156608512, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:34:772.145 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011156608512, Pid 113882]: SANDESH: Send FAILED: 1756692934771914 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.14 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:34:772.207 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011156608512, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:34:772.365 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011160806912, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:34:772.388 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011171994240, Pid 113882]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.14 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:15:34:772.540 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011160806912, Pid 113882]: SANDESH: Send FAILED: 1756692934772296 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.14 state = Connect event = Connect Event local_port = 42767 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:34:772.661 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011160806912, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:34:772.719 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:15:34:772.962 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011156608512, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:34:773.026 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011156608512, Pid 113882]: SANDESH: Send FAILED: 1756692934772827 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.14 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:34:773.089 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011156608512, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:34:773.122 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011156608512, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:34:773.166 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011156608512, Pid 113882]: SANDESH: Send FAILED: 1756692934772908 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.14 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:34:773.215 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011156608512, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:38:585.655 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011171994240, Pid 113882]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.0.0.56:8085::10.0.0.56:51628 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 255 2025-09-01 Mon 02:15:41:316.394 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011171994240, Pid 113882]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.20.0.14:8085::10.20.0.14:57928 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 255 2025-09-01 Mon 02:15:41:329.572 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011171994240, Pid 113882]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.20.0.14:8085::10.20.0.14:57940 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 255 2025-09-01 Mon 02:15:51:630.206 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011171994240, Pid 113882]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.0.0.56:8085::10.0.0.56:58584 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 255 2025-09-01 Mon 02:15:51:646.543 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:51:646.680 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: SANDESH: Send FAILED: 1756692951646397 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.14 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:51:646.717 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:51:646.827 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:51:646.830 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011171994240, Pid 113882]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.14 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:15:51:646.906 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: SANDESH: Send FAILED: 1756692951646728 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.14 state = Connect event = Connect Event local_port = 38537 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:51:647.086 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:51:647.216 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011160806912, Pid 113882]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:15:51:647.376 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011139814912, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:51:647.467 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011139814912, Pid 113882]: SANDESH: Send FAILED: 1756692951647339 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.14 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:51:647.495 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011139814912, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:51:647.520 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011139814912, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:51:647.549 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011139814912, Pid 113882]: SANDESH: Send FAILED: 1756692951647370 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.14 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:51:647.567 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011139814912, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:54:416.485 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011152410112, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:54:416.604 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011152410112, Pid 113882]: SANDESH: Send FAILED: 1756692954416258 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3993-1:Compute:contrail-vrouter-agent:0 sm_queue_count = 0 sm_stats= [ ev_stats= [ [ [ event = scm::EvCollectorUpdate enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvIdleHoldTimerExpired enqueues = 27 dequeues = 27 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvSandeshSend enqueues = 466 dequeues = 466 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvStart enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpConnectFail enqueues = 28 dequeues = 28 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpDeleteSession enqueues = 28 dequeues = 28 enqueue_fails = 0 dequeue_fails = 0 ], [ event = enqueues = 551 dequeues = 551 enqueue_fails = 0 dequeue_fails = 0 ], ] ] state = Idle state_since = 1756692925439081 last_state = Idle last_event = scm::EvSandeshSend last_event_at = 1756692951647516 ] max_sm_queue_count = 5 ] 2025-09-01 Mon 02:15:54:416.625 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011152410112, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:15:54:416.640 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011152410112, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:54:416.925 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011152410112, Pid 113882]: SANDESH: Send FAILED: 1756692954416402 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3993-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Idle successful_connections = 0 pid = 113882 http_port = 8085 start_time = 1756692234402667 collector_name = collector_ip = 10.0.0.56:8086 collector_list= [ [ (*_iter40) = 10.0.0.56:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ComputeCpuStateTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 11 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 61 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 104 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 165 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterControlStatsTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 23 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 34 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 228 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 58 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 226, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 466, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:15:54:416.965 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011152410112, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:15:54:423.417 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011160806912, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:54:423.490 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011160806912, Pid 113882]: SANDESH: Send FAILED: 1756692954423382 [SYS_NOTICE]: VrouterStats: data= [ name = cn-jenkins-deploy-platform-ansible-os-3993-1 cpu_info= [ num_cpu = 8 sys_mem_info= [ total = 32 used = 0 free = 0 buffers = 0 cached = 3882065592 node_type = ] meminfo= [ virt = 32598 peakvirt = 0 res = 0 ] cpuload= [ one_min_avg = 0.21 five_min_avg = 0.19875 fifteen_min_avg = 0.21875 ] cpu_share = 0.291667 ] ] 2025-09-01 Mon 02:15:54:423.501 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011160806912, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: VrouterStats 2025-09-01 Mon 02:15:54:423.526 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011160806912, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:54:423.544 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011160806912, Pid 113882]: SANDESH: Send FAILED: 1756692954423399 [SYS_NOTICE]: ComputeCpuStateTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3993-1 cpu_info= [ [ [ mem_virt = 32598 cpu_share = 0.291667 used_sys_mem = 412901704 one_min_cpuload = 0.21 mem_res = 0 ], ] ] ] 2025-09-01 Mon 02:15:54:423.551 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011160806912, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: ComputeCpuStateTrace 2025-09-01 Mon 02:15:54:423.560 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011160806912, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:54:423.673 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011160806912, Pid 113882]: SANDESH: Send FAILED: 1756692954423425 [SYS_NOTICE]: VrouterStats: data= [ name = cn-jenkins-deploy-platform-ansible-os-3993-1 in_tpkts = 199 in_bytes = 8554 out_tpkts = 202 out_bytes = 8764 raw_vhost_stats= [ name = vhost0 in_pkts = 199 in_bytes = 8554 out_pkts = 202 out_bytes = 8764 speed = -1 duplexity = -1 ] raw_vhost_drop_stats= [ ds_discard = 0 ds_pull = 0 ds_invalid_if = 0 ds_invalid_arp = 0 ds_trap_no_if = 0 ds_nowhere_to_go = 0 ds_flow_queue_limit_exceeded = 0 ds_flow_no_memory = 0 ds_flow_invalid_protocol = 0 ds_flow_nat_no_rflow = 0 ds_flow_action_drop = 0 ds_flow_action_invalid = 0 ds_flow_unusable = 0 ds_flow_table_full = 0 ds_interface_tx_discard = 0 ds_interface_drop = 0 ds_duplicated = 0 ds_push = 0 ds_ttl_exceeded = 0 ds_invalid_nh = 0 ds_invalid_label = 0 ds_invalid_protocol = 0 ds_interface_rx_discard = 0 ds_invalid_mcast_source = 0 ds_head_alloc_fail = 0 ds_pcow_fail = 0 ds_mcast_clone_fail = 0 ds_rewrite_fail = 0 ds_misc = 0 ds_invalid_packet = 0 ds_cksum_err = 0 ds_no_fmd = 0 ds_invalid_vnid = 0 ds_frag_err = 0 ds_invalid_source = 0 ds_mcast_df_bit = 0 ds_l2_no_route = 0 ds_vlan_fwd_tx = 0 ds_vlan_fwd_enq = 0 ds_drop_new_flow = 0 ds_flow_evict = 0 ds_trap_original = 0 ds_no_memory = 0 ds_fragment_queue_fail = 0 ds_no_frag_entry = 0 ds_icmp_error = 0 ds_clone_fail = 0 ds_drop_pkts = 0 ] raw_drop_stats= [ ds_discard = 0 ds_pull = 0 ds_invalid_if = 0 ds_invalid_arp = 0 ds_trap_no_if = 0 ds_nowhere_to_go = 0 ds_flow_queue_limit_exceeded = 0 ds_flow_no_memory = 0 ds_flow_invalid_protocol = 0 ds_flow_nat_no_rflow = 0 ds_flow_action_drop = 0 ds_flow_action_invalid = 0 ds_flow_unusable = 0 ds_flow_table_full = 0 ds_interface_tx_discard = 0 ds_interface_drop = 0 ds_duplicated = 0 ds_push = 0 ds_ttl_exceeded = 0 ds_invalid_nh = 0 ds_invalid_label = 0 ds_invalid_protocol = 0 ds_interface_rx_discard = 0 ds_invalid_mcast_source = 0 ds_head_alloc_fail = 0 ds_pcow_fail = 0 ds_mcast_clone_fail = 0 ds_rewrite_fail = 0 ds_misc = 0 ds_invalid_packet = 0 ds_cksum_err = 0 ds_no_fmd = 0 ds_invalid_vnid = 0 ds_frag_err = 0 ds_invalid_source = 0 ds_mcast_df_bit = 0 ds_l2_no_route = 0 ds_vlan_fwd_tx = 0 ds_vlan_fwd_enq = 0 ds_drop_new_flow = 0 ds_flow_evict = 0 ds_trap_original = 0 ds_no_memory = 0 ds_fragment_queue_fail = 0 ds_no_frag_entry = 0 ds_icmp_error = 0 ds_clone_fail = 0 ds_drop_pkts = 0 ] flow_rate= [ added_flows = 0 max_flow_adds_per_second = 0 min_flow_adds_per_second = 0 deleted_flows = 0 max_flow_deletes_per_second = 0 min_flow_deletes_per_second = 0 active_flows = 0 hold_flows = 0 ] raw_phy_if_stats= [ [ _iter262->first = ens6 [ in_pkts = 10 in_bytes = 700 out_pkts = 199 out_bytes = 8554 ], ] ] raw_phy_if_drop_stats= [ [ _iter263->first = ens6 [ ds_discard = 0 ds_pull = 0 ds_invalid_if = 0 ds_invalid_arp = 0 ds_trap_no_if = 0 ds_nowhere_to_go = 0 ds_flow_queue_limit_exceeded = 0 ds_flow_no_memory = 0 ds_flow_invalid_protocol = 0 ds_flow_nat_no_rflow = 0 ds_flow_action_drop = 0 ds_flow_action_invalid = 0 ds_flow_unusable = 0 ds_flow_table_full = 0 ds_interface_tx_discard = 0 ds_interface_drop = 0 ds_duplicated = 0 ds_push = 0 ds_ttl_exceeded = 0 ds_invalid_nh = 0 ds_invalid_label = 0 ds_invalid_protocol = 0 ds_interface_rx_discard = 0 ds_invalid_mcast_source = 0 ds_head_alloc_fail = 0 ds_pcow_fail = 0 ds_mcast_clone_fail = 0 ds_rewrite_fail = 0 ds_misc = 0 ds_invalid_packet = 0 ds_cksum_err = 0 ds_no_fmd = 0 ds_invalid_vnid = 0 ds_frag_err = 0 ds_invalid_source = 0 ds_mcast_df_bit = 0 ds_l2_no_route = 0 ds_vlan_fwd_tx = 0 ds_vlan_fwd_enq = 0 ds_drop_new_flow = 0 ds_flow_evict = 0 ds_trap_original = 0 ds_no_memory = 0 ds_fragment_queue_fail = 0 ds_no_frag_entry = 0 ds_icmp_error = 0 ds_clone_fail = 0 ds_drop_pkts = 0 ], ] ] ] 2025-09-01 Mon 02:15:54:423.694 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011160806912, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: VrouterStats 2025-09-01 Mon 02:15:54:423.702 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011160806912, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:54:423.728 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011160806912, Pid 113882]: SANDESH: Send FAILED: 1756692954423453 [SYS_NOTICE]: VrouterControlStatsTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3993-1 raw_xmpp_stats= [ [ _iter306->first = 10.20.0.14 [ reconnects = 0 in_msgs = 0 out_msgs = 0 ], ] ] raw_ifmap_stats= [ [ _iter307->first = link_delete_parse_errors _iter307->second = 0, _iter307->first = link_update_parse_errors _iter307->second = 0, _iter307->first = node_delete_parse_errors _iter307->second = 0, _iter307->first = node_update_parse_errors _iter307->second = 0, ] ] raw_rt_table_size= [ [ _iter308->first = default-domain:default-project:ip-fabric:__default__ [ inet4_unicast = 4 inet4_multicast = 0 evpn = 0 bridge = 2 inet6_unicast = 2 ], _iter308->first = default-domain:default-project:ip-fabric:ip-fabric [ inet4_unicast = 2 inet4_multicast = 0 evpn = 1 bridge = 2 inet6_unicast = 0 ], ] ] ] 2025-09-01 Mon 02:15:54:423.734 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011160806912, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: VrouterControlStatsTrace 2025-09-01 Mon 02:15:55:439.365 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011156608512, Pid 113882]: Processing scm::EvIdleHoldTimerExpired in state Idle 2025-09-01 Mon 02:15:55:439.880 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011156608512, Pid 113882]: Connect : Start Connect timer 10.0.0.56:8086 2025-09-01 Mon 02:15:55:439.991 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011171994240, Pid 113882]: -(-1) OnSessionEvent TCP Connect Failed 2025-09-01 Mon 02:15:55:440.079 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011171994240, Pid 113882]: -(-1):EvTcpConnectFail 2025-09-01 Mon 02:15:55:440.112 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011156608512, Pid 113882]: Processing scm::EvSandeshSend in state Connect 2025-09-01 Mon 02:15:55:440.207 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011156608512, Pid 113882]: SANDESH: Send FAILED: 1756692955439550 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-3993-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.14 server_addrs= [ [ (*_iter6) = 10.20.0.14:5269, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.14 server_addrs= [ [ (*_iter6) = 10.20.0.14:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.56:8086, ] ] status = Initializing description = Idle : EvIdleHoldTimerExpired -> Connect ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.14, XMPP:dns-server:10.20.0.14, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:15:55:440.256 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011156608512, Pid 113882]: Wrong state: Connect for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:15:55:440.283 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011156608512, Pid 113882]: Processing scm::EvSandeshSend in state Connect 2025-09-01 Mon 02:15:55:440.787 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011156608512, Pid 113882]: SANDESH: Send FAILED: 1756692955440071 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3993-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Connect successful_connections = 0 pid = 113882 http_port = 8085 start_time = 1756692234402667 collector_name = collector_ip = 10.0.0.56:8086 collector_list= [ [ (*_iter40) = 10.0.0.56:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ComputeCpuStateTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 12 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 61 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 106 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 165 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterControlStatsTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 24 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 36 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 228 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 58 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 226, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 472, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:15:55:440.850 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011156608512, Pid 113882]: Wrong state: Connect for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:15:55:440.880 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011156608512, Pid 113882]: Processing scm::EvTcpConnectFail in state Connect 2025-09-01 Mon 02:15:55:440.900 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011156608512, Pid 113882]: Connect : EvTcpConnectFail 2025-09-01 Mon 02:15:55:441.022 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011156608512, Pid 113882]: TCP [SYS_ERR]: TcpSessionMessageLog: Session -(-1) > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:15:55:441.107 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011156608512, Pid 113882]: Idle 2025-09-01 Mon 02:15:55:441.290 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011156608512, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:55:441.348 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011156608512, Pid 113882]: SANDESH: Send FAILED: 1756692955440958 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-3993-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.14 server_addrs= [ [ (*_iter6) = 10.20.0.14:5269, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.14 server_addrs= [ [ (*_iter6) = 10.20.0.14:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.14, XMPP:dns-server:10.20.0.14, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:15:55:441.381 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011156608512, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:15:55:441.404 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011156608512, Pid 113882]: Processing scm::EvTcpDeleteSession in state Idle 2025-09-01 Mon 02:15:55:441.477 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011156608512, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:55:441.906 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011156608512, Pid 113882]: SANDESH: Send FAILED: 1756692955441259 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3993-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Idle successful_connections = 0 pid = 113882 http_port = 8085 start_time = 1756692234402667 collector_name = collector_ip = 10.0.0.56:8086 collector_list= [ [ (*_iter40) = 10.0.0.56:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ComputeCpuStateTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 12 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 62 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 107 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 166 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterControlStatsTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 24 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 36 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 228 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 58 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 227, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 474, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:15:55:441.956 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011156608512, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:16:05:013.198 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011160806912, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:16:05:013.274 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011160806912, Pid 113882]: SANDESH: Send FAILED: 1756692965013149 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.14 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:16:05:013.290 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011160806912, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:16:05:013.374 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011152410112, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:16:05:013.425 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011152410112, Pid 113882]: SANDESH: Send FAILED: 1756692965013339 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.14 state = Connect event = Connect Event local_port = 33231 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:16:05:013.443 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011152410112, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:16:05:013.423 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011171994240, Pid 113882]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.14 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:16:05:013.615 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011139814912, Pid 113882]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:16:05:013.667 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011156608512, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:16:05:013.699 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011156608512, Pid 113882]: SANDESH: Send FAILED: 1756692965013653 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.14 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:16:05:013.709 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011156608512, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:16:05:013.736 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011156608512, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:16:05:013.751 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011156608512, Pid 113882]: SANDESH: Send FAILED: 1756692965013662 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.14 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:16:05:013.758 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011156608512, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:16:05:072.479 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011171994240, Pid 113882]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.0.0.56:8085::10.0.0.56:45416 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 255 2025-09-01 Mon 02:16:17:993.796 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011171994240, Pid 113882]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.0.0.56:8085::10.0.0.56:37940 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 255 2025-09-01 Mon 02:16:21:887.760 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011148211712, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:16:21:887.889 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011148211712, Pid 113882]: SANDESH: Send FAILED: 1756692981887690 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.14 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:16:21:887.917 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011148211712, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:16:21:888.133 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011171994240, Pid 113882]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.14 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:16:21:888.134 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:16:21:888.336 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011148211712, Pid 113882]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:16:21:888.371 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: SANDESH: Send FAILED: 1756692981888059 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.14 state = Connect event = Connect Event local_port = 37687 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:16:21:888.400 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:16:21:888.544 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:16:21:888.605 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: SANDESH: Send FAILED: 1756692981888415 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.14 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:16:21:888.633 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:16:21:888.680 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:16:21:888.708 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: SANDESH: Send FAILED: 1756692981888466 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.14 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:16:21:888.726 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:16:24:417.076 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:16:24:417.267 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: SANDESH: Send FAILED: 1756692984416782 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3993-1:Compute:contrail-vrouter-agent:0 sm_queue_count = 0 sm_stats= [ ev_stats= [ [ [ event = scm::EvCollectorUpdate enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvIdleHoldTimerExpired enqueues = 28 dequeues = 28 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvSandeshSend enqueues = 484 dequeues = 484 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvStart enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpConnectFail enqueues = 29 dequeues = 29 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpDeleteSession enqueues = 29 dequeues = 29 enqueue_fails = 0 dequeue_fails = 0 ], [ event = enqueues = 572 dequeues = 572 enqueue_fails = 0 dequeue_fails = 0 ], ] ] state = Idle state_since = 1756692955441103 last_state = Idle last_event = scm::EvSandeshSend last_event_at = 1756692981888676 ] max_sm_queue_count = 5 ] 2025-09-01 Mon 02:16:24:417.304 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:16:24:417.330 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:16:24:417.716 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: SANDESH: Send FAILED: 1756692984416970 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3993-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Idle successful_connections = 0 pid = 113882 http_port = 8085 start_time = 1756692234402667 collector_name = collector_ip = 10.0.0.56:8086 collector_list= [ [ (*_iter40) = 10.0.0.56:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ComputeCpuStateTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 12 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 63 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 108 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 170 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterControlStatsTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 24 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 36 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 236 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 60 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 233, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 484, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:16:24:417.776 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:16:24:423.850 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011135616512, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:16:24:424.334 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011135616512, Pid 113882]: SANDESH: Send FAILED: 1756692984423785 [SYS_NOTICE]: VrouterStats: data= [ name = cn-jenkins-deploy-platform-ansible-os-3993-1 in_tpkts = 202 in_bytes = 8680 out_tpkts = 205 out_bytes = 8890 raw_vhost_stats= [ name = vhost0 in_pkts = 202 in_bytes = 8680 out_pkts = 205 out_bytes = 8890 speed = -1 duplexity = -1 ] raw_vhost_drop_stats= [ ds_discard = 0 ds_pull = 0 ds_invalid_if = 0 ds_invalid_arp = 0 ds_trap_no_if = 0 ds_nowhere_to_go = 0 ds_flow_queue_limit_exceeded = 0 ds_flow_no_memory = 0 ds_flow_invalid_protocol = 0 ds_flow_nat_no_rflow = 0 ds_flow_action_drop = 0 ds_flow_action_invalid = 0 ds_flow_unusable = 0 ds_flow_table_full = 0 ds_interface_tx_discard = 0 ds_interface_drop = 0 ds_duplicated = 0 ds_push = 0 ds_ttl_exceeded = 0 ds_invalid_nh = 0 ds_invalid_label = 0 ds_invalid_protocol = 0 ds_interface_rx_discard = 0 ds_invalid_mcast_source = 0 ds_head_alloc_fail = 0 ds_pcow_fail = 0 ds_mcast_clone_fail = 0 ds_rewrite_fail = 0 ds_misc = 0 ds_invalid_packet = 0 ds_cksum_err = 0 ds_no_fmd = 0 ds_invalid_vnid = 0 ds_frag_err = 0 ds_invalid_source = 0 ds_mcast_df_bit = 0 ds_l2_no_route = 0 ds_vlan_fwd_tx = 0 ds_vlan_fwd_enq = 0 ds_drop_new_flow = 0 ds_flow_evict = 0 ds_trap_original = 0 ds_no_memory = 0 ds_fragment_queue_fail = 0 ds_no_frag_entry = 0 ds_icmp_error = 0 ds_clone_fail = 0 ds_drop_pkts = 0 ] raw_drop_stats= [ ds_discard = 0 ds_pull = 0 ds_invalid_if = 0 ds_invalid_arp = 0 ds_trap_no_if = 0 ds_nowhere_to_go = 0 ds_flow_queue_limit_exceeded = 0 ds_flow_no_memory = 0 ds_flow_invalid_protocol = 0 ds_flow_nat_no_rflow = 0 ds_flow_action_drop = 0 ds_flow_action_invalid = 0 ds_flow_unusable = 0 ds_flow_table_full = 0 ds_interface_tx_discard = 0 ds_interface_drop = 0 ds_duplicated = 0 ds_push = 0 ds_ttl_exceeded = 0 ds_invalid_nh = 0 ds_invalid_label = 0 ds_invalid_protocol = 0 ds_interface_rx_discard = 0 ds_invalid_mcast_source = 0 ds_head_alloc_fail = 0 ds_pcow_fail = 0 ds_mcast_clone_fail = 0 ds_rewrite_fail = 0 ds_misc = 0 ds_invalid_packet = 0 ds_cksum_err = 0 ds_no_fmd = 0 ds_invalid_vnid = 0 ds_frag_err = 0 ds_invalid_source = 0 ds_mcast_df_bit = 0 ds_l2_no_route = 0 ds_vlan_fwd_tx = 0 ds_vlan_fwd_enq = 0 ds_drop_new_flow = 0 ds_flow_evict = 0 ds_trap_original = 0 ds_no_memory = 0 ds_fragment_queue_fail = 0 ds_no_frag_entry = 0 ds_icmp_error = 0 ds_clone_fail = 0 ds_drop_pkts = 0 ] flow_rate= [ added_flows = 0 max_flow_adds_per_second = 0 min_flow_adds_per_second = 0 deleted_flows = 0 max_flow_deletes_per_second = 0 min_flow_deletes_per_second = 0 active_flows = 0 hold_flows = 0 ] phy_band_in_bps= [ [ _iter260->first = ens6 _iter260->second = 0, ] ] phy_band_out_bps= [ [ _iter261->first = ens6 _iter261->second = 33, ] ] raw_phy_if_stats= [ [ _iter262->first = ens6 [ in_pkts = 10 in_bytes = 700 out_pkts = 202 out_bytes = 8680 ], ] ] raw_phy_if_drop_stats= [ [ _iter263->first = ens6 [ ds_discard = 0 ds_pull = 0 ds_invalid_if = 0 ds_invalid_arp = 0 ds_trap_no_if = 0 ds_nowhere_to_go = 0 ds_flow_queue_limit_exceeded = 0 ds_flow_no_memory = 0 ds_flow_invalid_protocol = 0 ds_flow_nat_no_rflow = 0 ds_flow_action_drop = 0 ds_flow_action_invalid = 0 ds_flow_unusable = 0 ds_flow_table_full = 0 ds_interface_tx_discard = 0 ds_interface_drop = 0 ds_duplicated = 0 ds_push = 0 ds_ttl_exceeded = 0 ds_invalid_nh = 0 ds_invalid_label = 0 ds_invalid_protocol = 0 ds_interface_rx_discard = 0 ds_invalid_mcast_source = 0 ds_head_alloc_fail = 0 ds_pcow_fail = 0 ds_mcast_clone_fail = 0 ds_rewrite_fail = 0 ds_misc = 0 ds_invalid_packet = 0 ds_cksum_err = 0 ds_no_fmd = 0 ds_invalid_vnid = 0 ds_frag_err = 0 ds_invalid_source = 0 ds_mcast_df_bit = 0 ds_l2_no_route = 0 ds_vlan_fwd_tx = 0 ds_vlan_fwd_enq = 0 ds_drop_new_flow = 0 ds_flow_evict = 0 ds_trap_original = 0 ds_no_memory = 0 ds_fragment_queue_fail = 0 ds_no_frag_entry = 0 ds_icmp_error = 0 ds_clone_fail = 0 ds_drop_pkts = 0 ], ] ] ] 2025-09-01 Mon 02:16:24:424.414 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011135616512, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: VrouterStats 2025-09-01 Mon 02:16:24:424.488 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011135616512, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:16:24:424.563 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011135616512, Pid 113882]: SANDESH: Send FAILED: 1756692984423875 [SYS_NOTICE]: VrouterControlStatsTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3993-1 raw_xmpp_stats= [ [ _iter306->first = 10.20.0.14 [ reconnects = 0 in_msgs = 0 out_msgs = 0 ], ] ] raw_ifmap_stats= [ [ _iter307->first = link_delete_parse_errors _iter307->second = 0, _iter307->first = link_update_parse_errors _iter307->second = 0, _iter307->first = node_delete_parse_errors _iter307->second = 0, _iter307->first = node_update_parse_errors _iter307->second = 0, ] ] raw_rt_table_size= [ [ _iter308->first = default-domain:default-project:ip-fabric:__default__ [ inet4_unicast = 4 inet4_multicast = 0 evpn = 0 bridge = 2 inet6_unicast = 2 ], _iter308->first = default-domain:default-project:ip-fabric:ip-fabric [ inet4_unicast = 2 inet4_multicast = 0 evpn = 1 bridge = 2 inet6_unicast = 0 ], ] ] ] 2025-09-01 Mon 02:16:24:424.594 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011135616512, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: VrouterControlStatsTrace 2025-09-01 Mon 02:16:25:441.304 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011148211712, Pid 113882]: Processing scm::EvIdleHoldTimerExpired in state Idle 2025-09-01 Mon 02:16:25:441.849 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011148211712, Pid 113882]: Connect : Start Connect timer 10.0.0.56:8086 2025-09-01 Mon 02:16:25:441.868 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011171994240, Pid 113882]: -(-1) OnSessionEvent TCP Connect Failed 2025-09-01 Mon 02:16:25:441.949 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011171994240, Pid 113882]: -(-1):EvTcpConnectFail 2025-09-01 Mon 02:16:25:442.113 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011148211712, Pid 113882]: Processing scm::EvSandeshSend in state Connect 2025-09-01 Mon 02:16:25:442.202 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011148211712, Pid 113882]: SANDESH: Send FAILED: 1756692985441476 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-3993-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.14 server_addrs= [ [ (*_iter6) = 10.20.0.14:5269, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.14 server_addrs= [ [ (*_iter6) = 10.20.0.14:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.56:8086, ] ] status = Initializing description = Idle : EvIdleHoldTimerExpired -> Connect ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.14, XMPP:dns-server:10.20.0.14, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:16:25:442.227 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011148211712, Pid 113882]: Wrong state: Connect for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:16:25:442.254 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011148211712, Pid 113882]: Processing scm::EvTcpConnectFail in state Connect 2025-09-01 Mon 02:16:25:442.274 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011148211712, Pid 113882]: Connect : EvTcpConnectFail 2025-09-01 Mon 02:16:25:442.415 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011148211712, Pid 113882]: TCP [SYS_ERR]: TcpSessionMessageLog: Session -(-1) > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:16:25:442.498 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011148211712, Pid 113882]: Idle 2025-09-01 Mon 02:16:25:442.645 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011148211712, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:16:25:443.041 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011148211712, Pid 113882]: SANDESH: Send FAILED: 1756692985442058 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3993-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Connect successful_connections = 0 pid = 113882 http_port = 8085 start_time = 1756692234402667 collector_name = collector_ip = 10.0.0.56:8086 collector_list= [ [ (*_iter40) = 10.0.0.56:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ComputeCpuStateTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 12 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 63 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 110 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 170 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterControlStatsTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 25 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 37 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 236 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 60 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 233, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 488, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:16:25:443.112 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011148211712, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:16:25:443.148 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011148211712, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:16:25:443.227 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011148211712, Pid 113882]: SANDESH: Send FAILED: 1756692985442346 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-3993-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.14 server_addrs= [ [ (*_iter6) = 10.20.0.14:5269, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.14 server_addrs= [ [ (*_iter6) = 10.20.0.14:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.14, XMPP:dns-server:10.20.0.14, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:16:25:443.247 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011148211712, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:16:25:443.270 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011148211712, Pid 113882]: Processing scm::EvTcpDeleteSession in state Idle 2025-09-01 Mon 02:16:25:443.332 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011148211712, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:16:25:443.727 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011148211712, Pid 113882]: SANDESH: Send FAILED: 1756692985442610 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3993-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Idle successful_connections = 0 pid = 113882 http_port = 8085 start_time = 1756692234402667 collector_name = collector_ip = 10.0.0.56:8086 collector_list= [ [ (*_iter40) = 10.0.0.56:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ComputeCpuStateTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 12 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 64 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 110 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 171 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterControlStatsTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 25 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 37 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 236 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 60 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 234, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 489, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:16:25:443.773 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011148211712, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:16:30:906.412 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011171994240, Pid 113882]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.0.0.56:8085::10.0.0.56:42684 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 255 2025-09-01 Mon 02:16:34:983.994 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011148211712, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:16:34:984.118 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011148211712, Pid 113882]: SANDESH: Send FAILED: 1756692994983936 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.14 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:16:34:984.145 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011148211712, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:16:34:984.351 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011171994240, Pid 113882]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.14 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:16:34:984.353 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011160806912, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:16:34:984.578 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011148211712, Pid 113882]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:16:34:984.595 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011160806912, Pid 113882]: SANDESH: Send FAILED: 1756692994984280 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.14 state = Connect event = Connect Event local_port = 41897 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:16:34:984.652 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011160806912, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:16:34:984.683 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011160806912, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:16:34:984.727 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011160806912, Pid 113882]: SANDESH: Send FAILED: 1756692994984659 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.14 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:16:34:984.755 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011160806912, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:16:34:984.785 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011160806912, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:16:34:984.830 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011160806912, Pid 113882]: SANDESH: Send FAILED: 1756692994984673 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.14 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:16:34:984.851 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011160806912, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:16:42:217.408 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011171994240, Pid 113882]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.20.0.14:8085::10.20.0.14:55736 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 255 2025-09-01 Mon 02:16:42:232.591 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011171994240, Pid 113882]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.20.0.14:8085::10.20.0.14:55742 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 255 2025-09-01 Mon 02:16:43:931.986 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011171994240, Pid 113882]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.0.0.56:8085::10.0.0.56:60926 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 255 2025-09-01 Mon 02:16:52:668.921 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011152410112, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:16:52:669.066 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011152410112, Pid 113882]: SANDESH: Send FAILED: 1756693012668853 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.14 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:16:52:669.129 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011152410112, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:16:52:669.294 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011171994240, Pid 113882]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.14 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:16:52:669.304 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011160806912, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:16:52:669.464 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011160806912, Pid 113882]: SANDESH: Send FAILED: 1756693012669226 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.14 state = Connect event = Connect Event local_port = 45793 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:16:52:669.487 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011160806912, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:16:52:669.495 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011139814912, Pid 113882]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:16:52:669.726 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011135616512, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:16:52:669.811 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011135616512, Pid 113882]: SANDESH: Send FAILED: 1756693012669612 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.14 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:16:52:669.838 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011135616512, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:16:52:669.863 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011135616512, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:16:52:669.892 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011135616512, Pid 113882]: SANDESH: Send FAILED: 1756693012669674 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.14 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:16:52:669.910 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011135616512, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:16:54:417.566 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011148211712, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:16:54:417.780 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011148211712, Pid 113882]: SANDESH: Send FAILED: 1756693014417250 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3993-1:Compute:contrail-vrouter-agent:0 sm_queue_count = 0 sm_stats= [ ev_stats= [ [ [ event = scm::EvCollectorUpdate enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvIdleHoldTimerExpired enqueues = 29 dequeues = 29 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvSandeshSend enqueues = 500 dequeues = 500 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvStart enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpConnectFail enqueues = 30 dequeues = 30 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpDeleteSession enqueues = 30 dequeues = 30 enqueue_fails = 0 dequeue_fails = 0 ], [ event = enqueues = 591 dequeues = 591 enqueue_fails = 0 dequeue_fails = 0 ], ] ] state = Idle state_since = 1756692985442495 last_state = Idle last_event = scm::EvSandeshSend last_event_at = 1756693012669859 ] max_sm_queue_count = 5 ] 2025-09-01 Mon 02:16:54:417.843 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011148211712, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:16:54:417.870 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011148211712, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:16:54:418.290 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011148211712, Pid 113882]: SANDESH: Send FAILED: 1756693014417458 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3993-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Idle successful_connections = 0 pid = 113882 http_port = 8085 start_time = 1756692234402667 collector_name = collector_ip = 10.0.0.56:8086 collector_list= [ [ (*_iter40) = 10.0.0.56:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ComputeCpuStateTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 12 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 65 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 112 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 177 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterControlStatsTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 25 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 37 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 244 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 62 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 242, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 500, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:16:54:418.340 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011148211712, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:16:54:424.247 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011139814912, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:16:54:424.357 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011139814912, Pid 113882]: SANDESH: Send FAILED: 1756693014424157 [SYS_NOTICE]: VrouterStats: data= [ name = cn-jenkins-deploy-platform-ansible-os-3993-1 cpu_info= [ num_cpu = 8 sys_mem_info= [ total = 32 used = 0 free = 0 buffers = 0 cached = 3882065592 node_type = ] meminfo= [ virt = 32598 peakvirt = 0 res = 0 ] cpuload= [ one_min_avg = 0.2125 five_min_avg = 0.205 fifteen_min_avg = 0.22 ] cpu_share = 0.310417 ] ] 2025-09-01 Mon 02:16:54:424.380 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011139814912, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: VrouterStats 2025-09-01 Mon 02:16:54:424.396 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011139814912, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:16:54:424.448 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011139814912, Pid 113882]: SANDESH: Send FAILED: 1756693014424230 [SYS_NOTICE]: ComputeCpuStateTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3993-1 cpu_info= [ [ [ mem_virt = 32598 cpu_share = 0.310417 used_sys_mem = 412901704 one_min_cpuload = 0.2125 mem_res = 0 ], ] ] ] 2025-09-01 Mon 02:16:54:424.464 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011139814912, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: ComputeCpuStateTrace 2025-09-01 Mon 02:16:54:424.478 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011139814912, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:16:54:424.722 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011139814912, Pid 113882]: SANDESH: Send FAILED: 1756693014424279 [SYS_NOTICE]: VrouterStats: data= [ name = cn-jenkins-deploy-platform-ansible-os-3993-1 in_tpkts = 205 in_bytes = 8806 out_tpkts = 208 out_bytes = 9016 raw_vhost_stats= [ name = vhost0 in_pkts = 205 in_bytes = 8806 out_pkts = 208 out_bytes = 9016 speed = -1 duplexity = -1 ] raw_vhost_drop_stats= [ ds_discard = 0 ds_pull = 0 ds_invalid_if = 0 ds_invalid_arp = 0 ds_trap_no_if = 0 ds_nowhere_to_go = 0 ds_flow_queue_limit_exceeded = 0 ds_flow_no_memory = 0 ds_flow_invalid_protocol = 0 ds_flow_nat_no_rflow = 0 ds_flow_action_drop = 0 ds_flow_action_invalid = 0 ds_flow_unusable = 0 ds_flow_table_full = 0 ds_interface_tx_discard = 0 ds_interface_drop = 0 ds_duplicated = 0 ds_push = 0 ds_ttl_exceeded = 0 ds_invalid_nh = 0 ds_invalid_label = 0 ds_invalid_protocol = 0 ds_interface_rx_discard = 0 ds_invalid_mcast_source = 0 ds_head_alloc_fail = 0 ds_pcow_fail = 0 ds_mcast_clone_fail = 0 ds_rewrite_fail = 0 ds_misc = 0 ds_invalid_packet = 0 ds_cksum_err = 0 ds_no_fmd = 0 ds_invalid_vnid = 0 ds_frag_err = 0 ds_invalid_source = 0 ds_mcast_df_bit = 0 ds_l2_no_route = 0 ds_vlan_fwd_tx = 0 ds_vlan_fwd_enq = 0 ds_drop_new_flow = 0 ds_flow_evict = 0 ds_trap_original = 0 ds_no_memory = 0 ds_fragment_queue_fail = 0 ds_no_frag_entry = 0 ds_icmp_error = 0 ds_clone_fail = 0 ds_drop_pkts = 0 ] raw_drop_stats= [ ds_discard = 0 ds_pull = 0 ds_invalid_if = 0 ds_invalid_arp = 0 ds_trap_no_if = 0 ds_nowhere_to_go = 0 ds_flow_queue_limit_exceeded = 0 ds_flow_no_memory = 0 ds_flow_invalid_protocol = 0 ds_flow_nat_no_rflow = 0 ds_flow_action_drop = 0 ds_flow_action_invalid = 0 ds_flow_unusable = 0 ds_flow_table_full = 0 ds_interface_tx_discard = 0 ds_interface_drop = 0 ds_duplicated = 0 ds_push = 0 ds_ttl_exceeded = 0 ds_invalid_nh = 0 ds_invalid_label = 0 ds_invalid_protocol = 0 ds_interface_rx_discard = 0 ds_invalid_mcast_source = 0 ds_head_alloc_fail = 0 ds_pcow_fail = 0 ds_mcast_clone_fail = 0 ds_rewrite_fail = 0 ds_misc = 0 ds_invalid_packet = 0 ds_cksum_err = 0 ds_no_fmd = 0 ds_invalid_vnid = 0 ds_frag_err = 0 ds_invalid_source = 0 ds_mcast_df_bit = 0 ds_l2_no_route = 0 ds_vlan_fwd_tx = 0 ds_vlan_fwd_enq = 0 ds_drop_new_flow = 0 ds_flow_evict = 0 ds_trap_original = 0 ds_no_memory = 0 ds_fragment_queue_fail = 0 ds_no_frag_entry = 0 ds_icmp_error = 0 ds_clone_fail = 0 ds_drop_pkts = 0 ] flow_rate= [ added_flows = 0 max_flow_adds_per_second = 0 min_flow_adds_per_second = 0 deleted_flows = 0 max_flow_deletes_per_second = 0 min_flow_deletes_per_second = 0 active_flows = 0 hold_flows = 0 ] raw_phy_if_stats= [ [ _iter262->first = ens6 [ in_pkts = 10 in_bytes = 700 out_pkts = 205 out_bytes = 8806 ], ] ] raw_phy_if_drop_stats= [ [ _iter263->first = ens6 [ ds_discard = 0 ds_pull = 0 ds_invalid_if = 0 ds_invalid_arp = 0 ds_trap_no_if = 0 ds_nowhere_to_go = 0 ds_flow_queue_limit_exceeded = 0 ds_flow_no_memory = 0 ds_flow_invalid_protocol = 0 ds_flow_nat_no_rflow = 0 ds_flow_action_drop = 0 ds_flow_action_invalid = 0 ds_flow_unusable = 0 ds_flow_table_full = 0 ds_interface_tx_discard = 0 ds_interface_drop = 0 ds_duplicated = 0 ds_push = 0 ds_ttl_exceeded = 0 ds_invalid_nh = 0 ds_invalid_label = 0 ds_invalid_protocol = 0 ds_interface_rx_discard = 0 ds_invalid_mcast_source = 0 ds_head_alloc_fail = 0 ds_pcow_fail = 0 ds_mcast_clone_fail = 0 ds_rewrite_fail = 0 ds_misc = 0 ds_invalid_packet = 0 ds_cksum_err = 0 ds_no_fmd = 0 ds_invalid_vnid = 0 ds_frag_err = 0 ds_invalid_source = 0 ds_mcast_df_bit = 0 ds_l2_no_route = 0 ds_vlan_fwd_tx = 0 ds_vlan_fwd_enq = 0 ds_drop_new_flow = 0 ds_flow_evict = 0 ds_trap_original = 0 ds_no_memory = 0 ds_fragment_queue_fail = 0 ds_no_frag_entry = 0 ds_icmp_error = 0 ds_clone_fail = 0 ds_drop_pkts = 0 ], ] ] ] 2025-09-01 Mon 02:16:54:424.757 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011139814912, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: VrouterStats 2025-09-01 Mon 02:16:54:424.772 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011139814912, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:16:54:424.822 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011139814912, Pid 113882]: SANDESH: Send FAILED: 1756693014424332 [SYS_NOTICE]: VrouterControlStatsTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3993-1 raw_xmpp_stats= [ [ _iter306->first = 10.20.0.14 [ reconnects = 0 in_msgs = 0 out_msgs = 0 ], ] ] raw_ifmap_stats= [ [ _iter307->first = link_delete_parse_errors _iter307->second = 0, _iter307->first = link_update_parse_errors _iter307->second = 0, _iter307->first = node_delete_parse_errors _iter307->second = 0, _iter307->first = node_update_parse_errors _iter307->second = 0, ] ] raw_rt_table_size= [ [ _iter308->first = default-domain:default-project:ip-fabric:__default__ [ inet4_unicast = 4 inet4_multicast = 0 evpn = 0 bridge = 2 inet6_unicast = 2 ], _iter308->first = default-domain:default-project:ip-fabric:ip-fabric [ inet4_unicast = 2 inet4_multicast = 0 evpn = 1 bridge = 2 inet6_unicast = 0 ], ] ] ] 2025-09-01 Mon 02:16:54:424.840 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011139814912, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: VrouterControlStatsTrace 2025-09-01 Mon 02:16:55:442.773 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011156608512, Pid 113882]: Processing scm::EvIdleHoldTimerExpired in state Idle 2025-09-01 Mon 02:16:55:443.340 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011156608512, Pid 113882]: Connect : Start Connect timer 10.0.0.56:8086 2025-09-01 Mon 02:16:55:443.397 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011171994240, Pid 113882]: -(-1) OnSessionEvent TCP Connect Failed 2025-09-01 Mon 02:16:55:443.470 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011171994240, Pid 113882]: -(-1):EvTcpConnectFail 2025-09-01 Mon 02:16:55:443.574 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011156608512, Pid 113882]: Processing scm::EvSandeshSend in state Connect 2025-09-01 Mon 02:16:55:443.643 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011156608512, Pid 113882]: SANDESH: Send FAILED: 1756693015442994 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-3993-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.14 server_addrs= [ [ (*_iter6) = 10.20.0.14:5269, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.14 server_addrs= [ [ (*_iter6) = 10.20.0.14:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.56:8086, ] ] status = Initializing description = Idle : EvIdleHoldTimerExpired -> Connect ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.14, XMPP:dns-server:10.20.0.14, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:16:55:443.668 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011156608512, Pid 113882]: Wrong state: Connect for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:16:55:443.694 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011156608512, Pid 113882]: Processing scm::EvTcpConnectFail in state Connect 2025-09-01 Mon 02:16:55:443.715 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011156608512, Pid 113882]: Connect : EvTcpConnectFail 2025-09-01 Mon 02:16:55:443.823 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011156608512, Pid 113882]: TCP [SYS_ERR]: TcpSessionMessageLog: Session -(-1) > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:16:55:443.892 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011156608512, Pid 113882]: Idle 2025-09-01 Mon 02:16:55:444.041 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011156608512, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:16:55:444.528 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011156608512, Pid 113882]: SANDESH: Send FAILED: 1756693015443523 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3993-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Connect successful_connections = 0 pid = 113882 http_port = 8085 start_time = 1756692234402667 collector_name = collector_ip = 10.0.0.56:8086 collector_list= [ [ (*_iter40) = 10.0.0.56:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ComputeCpuStateTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 13 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 65 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 114 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 177 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterControlStatsTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 26 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 39 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 244 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 62 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 242, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 506, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:16:55:444.607 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011156608512, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:16:55:444.637 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011156608512, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:16:55:444.687 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011156608512, Pid 113882]: SANDESH: Send FAILED: 1756693015443767 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-3993-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.14 server_addrs= [ [ (*_iter6) = 10.20.0.14:5269, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.14 server_addrs= [ [ (*_iter6) = 10.20.0.14:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.14, XMPP:dns-server:10.20.0.14, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:16:55:444.706 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011156608512, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:16:55:444.729 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011156608512, Pid 113882]: Processing scm::EvTcpDeleteSession in state Idle 2025-09-01 Mon 02:16:55:444.788 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011156608512, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:16:55:445.266 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011156608512, Pid 113882]: SANDESH: Send FAILED: 1756693015444008 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3993-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Idle successful_connections = 0 pid = 113882 http_port = 8085 start_time = 1756692234402667 collector_name = collector_ip = 10.0.0.56:8086 collector_list= [ [ (*_iter40) = 10.0.0.56:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ComputeCpuStateTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 13 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 66 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 114 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 178 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterControlStatsTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 26 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 39 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 244 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 62 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 243, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 507, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:16:55:445.333 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011156608512, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:16:56:892.311 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011171994240, Pid 113882]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.0.0.56:8085::10.0.0.56:56340 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 255 2025-09-01 Mon 02:17:04:145.132 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011152410112, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:17:04:145.285 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011152410112, Pid 113882]: SANDESH: Send FAILED: 1756693024145074 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.14 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:17:04:145.314 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011152410112, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:17:04:145.566 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011160806912, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:17:04:145.691 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011160806912, Pid 113882]: SANDESH: Send FAILED: 1756693024145478 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.14 state = Connect event = Connect Event local_port = 36929 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:17:04:145.753 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011160806912, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:17:04:145.581 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011171994240, Pid 113882]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.14 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:17:04:146.016 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011165005312, Pid 113882]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:17:04:146.250 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011139814912, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:17:04:146.342 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011139814912, Pid 113882]: SANDESH: Send FAILED: 1756693024146116 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.14 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:17:04:146.368 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011139814912, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:17:04:146.394 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011139814912, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:17:04:146.423 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011139814912, Pid 113882]: SANDESH: Send FAILED: 1756693024146159 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.14 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:17:04:146.441 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011139814912, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:17:10:351.803 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011171994240, Pid 113882]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.0.0.56:8085::10.0.0.56:45362 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 255 2025-09-01 Mon 02:17:13:340.674 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011171994240, Pid 113882]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.0.0.56:8085::10.0.0.56:33980 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 255 2025-09-01 Mon 02:17:16:263.953 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011171994240, Pid 113882]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.0.0.56:8085::10.0.0.56:33996 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 255 2025-09-01 Mon 02:17:22:370.047 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011148211712, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:17:22:370.134 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011148211712, Pid 113882]: SANDESH: Send FAILED: 1756693042369997 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.14 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:17:22:370.146 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011148211712, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:17:22:370.419 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011160806912, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:17:22:370.454 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011171994240, Pid 113882]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.14 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:17:22:370.547 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011160806912, Pid 113882]: SANDESH: Send FAILED: 1756693042370352 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.14 state = Connect event = Connect Event local_port = 45551 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:17:22:370.699 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011160806912, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:17:22:370.799 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011139814912, Pid 113882]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:17:22:371.021 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011156608512, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:17:22:371.113 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011156608512, Pid 113882]: SANDESH: Send FAILED: 1756693042370905 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.14 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:17:22:371.141 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011156608512, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:17:22:371.165 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011156608512, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:17:22:371.212 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011156608512, Pid 113882]: SANDESH: Send FAILED: 1756693042370958 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.14 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:17:22:371.231 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011156608512, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:17:24:417.785 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011135616512, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:17:24:417.852 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011135616512, Pid 113882]: SANDESH: Send FAILED: 1756693044417661 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3993-1:Compute:contrail-vrouter-agent:0 sm_queue_count = 0 sm_stats= [ ev_stats= [ [ [ event = scm::EvCollectorUpdate enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvIdleHoldTimerExpired enqueues = 30 dequeues = 30 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvSandeshSend enqueues = 518 dequeues = 518 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvStart enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpConnectFail enqueues = 31 dequeues = 31 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpDeleteSession enqueues = 31 dequeues = 31 enqueue_fails = 0 dequeue_fails = 0 ], [ event = enqueues = 612 dequeues = 612 enqueue_fails = 0 dequeue_fails = 0 ], ] ] state = Idle state_since = 1756693015443889 last_state = Idle last_event = scm::EvSandeshSend last_event_at = 1756693042371161 ] max_sm_queue_count = 5 ] 2025-09-01 Mon 02:17:24:417.863 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011135616512, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:17:24:417.881 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011135616512, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:17:24:417.995 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011135616512, Pid 113882]: SANDESH: Send FAILED: 1756693044417750 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3993-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Idle successful_connections = 0 pid = 113882 http_port = 8085 start_time = 1756692234402667 collector_name = collector_ip = 10.0.0.56:8086 collector_list= [ [ (*_iter40) = 10.0.0.56:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ComputeCpuStateTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 13 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 67 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 116 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 184 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterControlStatsTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 26 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 39 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 252 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 64 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 251, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 518, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:17:24:418.009 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011135616512, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:17:24:424.788 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011160806912, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:17:24:425.294 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011160806912, Pid 113882]: SANDESH: Send FAILED: 1756693044424689 [SYS_NOTICE]: VrouterStats: data= [ name = cn-jenkins-deploy-platform-ansible-os-3993-1 in_tpkts = 208 in_bytes = 8932 out_tpkts = 246 out_bytes = 10612 raw_vhost_stats= [ name = vhost0 in_pkts = 208 in_bytes = 8932 out_pkts = 246 out_bytes = 10612 speed = -1 duplexity = -1 ] raw_vhost_drop_stats= [ ds_discard = 0 ds_pull = 0 ds_invalid_if = 0 ds_invalid_arp = 0 ds_trap_no_if = 0 ds_nowhere_to_go = 0 ds_flow_queue_limit_exceeded = 0 ds_flow_no_memory = 0 ds_flow_invalid_protocol = 0 ds_flow_nat_no_rflow = 0 ds_flow_action_drop = 0 ds_flow_action_invalid = 0 ds_flow_unusable = 0 ds_flow_table_full = 0 ds_interface_tx_discard = 0 ds_interface_drop = 0 ds_duplicated = 0 ds_push = 0 ds_ttl_exceeded = 0 ds_invalid_nh = 0 ds_invalid_label = 0 ds_invalid_protocol = 0 ds_interface_rx_discard = 0 ds_invalid_mcast_source = 0 ds_head_alloc_fail = 0 ds_pcow_fail = 0 ds_mcast_clone_fail = 0 ds_rewrite_fail = 0 ds_misc = 0 ds_invalid_packet = 0 ds_cksum_err = 0 ds_no_fmd = 0 ds_invalid_vnid = 0 ds_frag_err = 0 ds_invalid_source = 0 ds_mcast_df_bit = 0 ds_l2_no_route = 0 ds_vlan_fwd_tx = 0 ds_vlan_fwd_enq = 0 ds_drop_new_flow = 0 ds_flow_evict = 0 ds_trap_original = 0 ds_no_memory = 0 ds_fragment_queue_fail = 0 ds_no_frag_entry = 0 ds_icmp_error = 0 ds_clone_fail = 0 ds_drop_pkts = 0 ] raw_drop_stats= [ ds_discard = 0 ds_pull = 0 ds_invalid_if = 0 ds_invalid_arp = 0 ds_trap_no_if = 0 ds_nowhere_to_go = 0 ds_flow_queue_limit_exceeded = 0 ds_flow_no_memory = 0 ds_flow_invalid_protocol = 0 ds_flow_nat_no_rflow = 0 ds_flow_action_drop = 0 ds_flow_action_invalid = 0 ds_flow_unusable = 0 ds_flow_table_full = 0 ds_interface_tx_discard = 0 ds_interface_drop = 0 ds_duplicated = 0 ds_push = 0 ds_ttl_exceeded = 0 ds_invalid_nh = 0 ds_invalid_label = 0 ds_invalid_protocol = 0 ds_interface_rx_discard = 0 ds_invalid_mcast_source = 0 ds_head_alloc_fail = 0 ds_pcow_fail = 0 ds_mcast_clone_fail = 0 ds_rewrite_fail = 0 ds_misc = 0 ds_invalid_packet = 0 ds_cksum_err = 0 ds_no_fmd = 0 ds_invalid_vnid = 0 ds_frag_err = 0 ds_invalid_source = 0 ds_mcast_df_bit = 0 ds_l2_no_route = 0 ds_vlan_fwd_tx = 0 ds_vlan_fwd_enq = 0 ds_drop_new_flow = 0 ds_flow_evict = 0 ds_trap_original = 0 ds_no_memory = 0 ds_fragment_queue_fail = 0 ds_no_frag_entry = 0 ds_icmp_error = 0 ds_clone_fail = 0 ds_drop_pkts = 0 ] total_in_bandwidth_utilization = 1.06252e-15 flow_rate= [ added_flows = 0 max_flow_adds_per_second = 0 min_flow_adds_per_second = 0 deleted_flows = 0 max_flow_deletes_per_second = 0 min_flow_deletes_per_second = 0 active_flows = 0 hold_flows = 0 ] phy_band_in_bps= [ [ _iter260->first = ens6 _iter260->second = 196, ] ] phy_band_out_bps= [ [ _iter261->first = ens6 _iter261->second = 33, ] ] raw_phy_if_stats= [ [ _iter262->first = ens6 [ in_pkts = 45 in_bytes = 2170 out_pkts = 208 out_bytes = 8932 ], ] ] raw_phy_if_drop_stats= [ [ _iter263->first = ens6 [ ds_discard = 0 ds_pull = 0 ds_invalid_if = 0 ds_invalid_arp = 0 ds_trap_no_if = 0 ds_nowhere_to_go = 0 ds_flow_queue_limit_exceeded = 0 ds_flow_no_memory = 0 ds_flow_invalid_protocol = 0 ds_flow_nat_no_rflow = 0 ds_flow_action_drop = 0 ds_flow_action_invalid = 0 ds_flow_unusable = 0 ds_flow_table_full = 0 ds_interface_tx_discard = 0 ds_interface_drop = 0 ds_duplicated = 0 ds_push = 0 ds_ttl_exceeded = 0 ds_invalid_nh = 0 ds_invalid_label = 0 ds_invalid_protocol = 0 ds_interface_rx_discard = 0 ds_invalid_mcast_source = 0 ds_head_alloc_fail = 0 ds_pcow_fail = 0 ds_mcast_clone_fail = 0 ds_rewrite_fail = 0 ds_misc = 0 ds_invalid_packet = 0 ds_cksum_err = 0 ds_no_fmd = 0 ds_invalid_vnid = 0 ds_frag_err = 0 ds_invalid_source = 0 ds_mcast_df_bit = 0 ds_l2_no_route = 0 ds_vlan_fwd_tx = 0 ds_vlan_fwd_enq = 0 ds_drop_new_flow = 0 ds_flow_evict = 0 ds_trap_original = 0 ds_no_memory = 0 ds_fragment_queue_fail = 0 ds_no_frag_entry = 0 ds_icmp_error = 0 ds_clone_fail = 0 ds_drop_pkts = 0 ], ] ] ] 2025-09-01 Mon 02:17:24:425.363 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011160806912, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: VrouterStats 2025-09-01 Mon 02:17:24:425.393 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011160806912, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:17:24:425.466 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011160806912, Pid 113882]: SANDESH: Send FAILED: 1756693044424807 [SYS_NOTICE]: VrouterControlStatsTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3993-1 raw_xmpp_stats= [ [ _iter306->first = 10.20.0.14 [ reconnects = 0 in_msgs = 0 out_msgs = 0 ], ] ] raw_ifmap_stats= [ [ _iter307->first = link_delete_parse_errors _iter307->second = 0, _iter307->first = link_update_parse_errors _iter307->second = 0, _iter307->first = node_delete_parse_errors _iter307->second = 0, _iter307->first = node_update_parse_errors _iter307->second = 0, ] ] raw_rt_table_size= [ [ _iter308->first = default-domain:default-project:ip-fabric:__default__ [ inet4_unicast = 4 inet4_multicast = 0 evpn = 0 bridge = 2 inet6_unicast = 2 ], _iter308->first = default-domain:default-project:ip-fabric:ip-fabric [ inet4_unicast = 2 inet4_multicast = 0 evpn = 1 bridge = 2 inet6_unicast = 0 ], ] ] ] 2025-09-01 Mon 02:17:24:425.486 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011160806912, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: VrouterControlStatsTrace 2025-09-01 Mon 02:17:25:444.157 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011139814912, Pid 113882]: Processing scm::EvIdleHoldTimerExpired in state Idle 2025-09-01 Mon 02:17:25:444.709 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011139814912, Pid 113882]: Connect : Start Connect timer 10.0.0.56:8086 2025-09-01 Mon 02:17:25:444.772 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011171994240, Pid 113882]: -(-1) OnSessionEvent TCP Connect Failed 2025-09-01 Mon 02:17:25:444.845 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011171994240, Pid 113882]: -(-1):EvTcpConnectFail 2025-09-01 Mon 02:17:25:444.938 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011139814912, Pid 113882]: Processing scm::EvSandeshSend in state Connect 2025-09-01 Mon 02:17:25:445.008 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011139814912, Pid 113882]: SANDESH: Send FAILED: 1756693045444363 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-3993-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.14 server_addrs= [ [ (*_iter6) = 10.20.0.14:5269, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.14 server_addrs= [ [ (*_iter6) = 10.20.0.14:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.56:8086, ] ] status = Initializing description = Idle : EvIdleHoldTimerExpired -> Connect ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.14, XMPP:dns-server:10.20.0.14, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:17:25:445.054 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011139814912, Pid 113882]: Wrong state: Connect for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:17:25:445.081 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011139814912, Pid 113882]: Processing scm::EvTcpConnectFail in state Connect 2025-09-01 Mon 02:17:25:445.102 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011139814912, Pid 113882]: Connect : EvTcpConnectFail 2025-09-01 Mon 02:17:25:445.229 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011139814912, Pid 113882]: TCP [SYS_ERR]: TcpSessionMessageLog: Session -(-1) > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:17:25:445.304 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011139814912, Pid 113882]: Idle 2025-09-01 Mon 02:17:25:445.473 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011139814912, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:17:25:446.009 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011139814912, Pid 113882]: SANDESH: Send FAILED: 1756693045444902 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3993-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Connect successful_connections = 0 pid = 113882 http_port = 8085 start_time = 1756692234402667 collector_name = collector_ip = 10.0.0.56:8086 collector_list= [ [ (*_iter40) = 10.0.0.56:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ComputeCpuStateTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 13 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 67 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 118 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 184 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterControlStatsTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 27 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 40 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 252 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 64 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 251, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 522, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:17:25:446.075 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011139814912, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:17:25:446.109 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011139814912, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:17:25:446.160 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011139814912, Pid 113882]: SANDESH: Send FAILED: 1756693045445156 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-3993-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.14 server_addrs= [ [ (*_iter6) = 10.20.0.14:5269, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.14 server_addrs= [ [ (*_iter6) = 10.20.0.14:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.14, XMPP:dns-server:10.20.0.14, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:17:25:446.201 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011139814912, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:17:25:446.225 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011139814912, Pid 113882]: Processing scm::EvTcpDeleteSession in state Idle 2025-09-01 Mon 02:17:25:446.289 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011139814912, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:17:25:446.667 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011139814912, Pid 113882]: SANDESH: Send FAILED: 1756693045445418 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3993-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Idle successful_connections = 0 pid = 113882 http_port = 8085 start_time = 1756692234402667 collector_name = collector_ip = 10.0.0.56:8086 collector_list= [ [ (*_iter40) = 10.0.0.56:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ComputeCpuStateTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 13 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 68 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 118 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 185 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterControlStatsTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 27 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 40 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 252 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 64 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 252, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 523, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:17:25:446.711 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011139814912, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:17:35:736.397 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011148211712, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:17:35:736.551 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011148211712, Pid 113882]: SANDESH: Send FAILED: 1756693055736339 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.14 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:17:35:736.579 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011148211712, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:17:35:736.588 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011171994240, Pid 113882]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.14 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:17:35:736.621 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011148211712, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:17:35:736.729 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011148211712, Pid 113882]: SANDESH: Send FAILED: 1756693055736543 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.14 state = Connect event = Connect Event local_port = 34445 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:17:35:736.741 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011135616512, Pid 113882]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:17:35:736.768 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011148211712, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:17:35:736.846 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011148211712, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:17:35:736.885 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011148211712, Pid 113882]: SANDESH: Send FAILED: 1756693055736810 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.14 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:17:35:736.913 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011148211712, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:17:35:736.946 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011148211712, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:17:35:736.990 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011148211712, Pid 113882]: SANDESH: Send FAILED: 1756693055736814 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.14 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:17:35:737.048 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011148211712, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:17:43:209.212 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011171994240, Pid 113882]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.20.0.14:8085::10.20.0.14:33980 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 255 2025-09-01 Mon 02:17:43:223.616 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011171994240, Pid 113882]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.20.0.14:8085::10.20.0.14:33982 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 255 2025-09-01 Mon 02:17:47:439.649 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011171994240, Pid 113882]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.0.0.56:8085::10.0.0.56:49540 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 255 2025-09-01 Mon 02:17:53:421.226 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011135616512, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:17:53:421.283 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011135616512, Pid 113882]: SANDESH: Send FAILED: 1756693073421192 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.14 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:17:53:421.292 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011135616512, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:17:53:421.497 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011148211712, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:17:53:421.510 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011171994240, Pid 113882]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.14 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:17:53:421.572 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011148211712, Pid 113882]: SANDESH: Send FAILED: 1756693073421454 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.14 state = Connect event = Connect Event local_port = 40131 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:17:53:421.644 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011148211712, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:17:53:421.654 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011135616512, Pid 113882]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:17:53:421.729 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011148211712, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:17:53:421.749 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011148211712, Pid 113882]: SANDESH: Send FAILED: 1756693073421686 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.14 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:17:53:421.759 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011148211712, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:17:53:421.771 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011148211712, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:17:53:421.785 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011148211712, Pid 113882]: SANDESH: Send FAILED: 1756693073421703 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.14 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:17:53:421.809 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011148211712, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:17:54:418.230 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011156608512, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:17:54:418.370 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011156608512, Pid 113882]: SANDESH: Send FAILED: 1756693074417959 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3993-1:Compute:contrail-vrouter-agent:0 sm_queue_count = 0 sm_stats= [ ev_stats= [ [ [ event = scm::EvCollectorUpdate enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvIdleHoldTimerExpired enqueues = 31 dequeues = 31 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvSandeshSend enqueues = 534 dequeues = 534 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvStart enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpConnectFail enqueues = 32 dequeues = 32 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpDeleteSession enqueues = 32 dequeues = 32 enqueue_fails = 0 dequeue_fails = 0 ], [ event = enqueues = 631 dequeues = 631 enqueue_fails = 0 dequeue_fails = 0 ], ] ] state = Idle state_since = 1756693045445301 last_state = Idle last_event = scm::EvSandeshSend last_event_at = 1756693073421769 ] max_sm_queue_count = 5 ] 2025-09-01 Mon 02:17:54:418.400 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011156608512, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:17:54:418.423 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011156608512, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:17:54:418.735 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011156608512, Pid 113882]: SANDESH: Send FAILED: 1756693074418125 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3993-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Idle successful_connections = 0 pid = 113882 http_port = 8085 start_time = 1756692234402667 collector_name = collector_ip = 10.0.0.56:8086 collector_list= [ [ (*_iter40) = 10.0.0.56:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ComputeCpuStateTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 13 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 69 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 120 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 190 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterControlStatsTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 27 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 40 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 260 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 66 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 259, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 534, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:17:54:418.793 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011156608512, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:17:54:425.084 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011160806912, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:17:54:425.276 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011160806912, Pid 113882]: SANDESH: Send FAILED: 1756693074425021 [SYS_NOTICE]: VrouterStats: data= [ name = cn-jenkins-deploy-platform-ansible-os-3993-1 cpu_info= [ num_cpu = 8 sys_mem_info= [ total = 32 used = 0 free = 0 buffers = 0 cached = 3882065592 node_type = ] meminfo= [ virt = 32598 peakvirt = 0 res = 0 ] cpuload= [ one_min_avg = 0.225 five_min_avg = 0.215 fifteen_min_avg = 0.22375 ] cpu_share = 0.3 ] ] 2025-09-01 Mon 02:17:54:425.308 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011160806912, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: VrouterStats 2025-09-01 Mon 02:17:54:425.337 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011160806912, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:17:54:425.413 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011160806912, Pid 113882]: SANDESH: Send FAILED: 1756693074425044 [SYS_NOTICE]: ComputeCpuStateTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3993-1 cpu_info= [ [ [ mem_virt = 32598 cpu_share = 0.3 used_sys_mem = 412901704 one_min_cpuload = 0.225 mem_res = 0 ], ] ] ] 2025-09-01 Mon 02:17:54:425.436 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011160806912, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: ComputeCpuStateTrace 2025-09-01 Mon 02:17:54:425.460 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011160806912, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:17:54:425.887 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011160806912, Pid 113882]: SANDESH: Send FAILED: 1756693074425079 [SYS_NOTICE]: VrouterStats: data= [ name = cn-jenkins-deploy-platform-ansible-os-3993-1 in_tpkts = 211 in_bytes = 9058 out_tpkts = 300 out_bytes = 12880 raw_vhost_stats= [ name = vhost0 in_pkts = 211 in_bytes = 9058 out_pkts = 300 out_bytes = 12880 speed = -1 duplexity = -1 ] raw_vhost_drop_stats= [ ds_discard = 0 ds_pull = 0 ds_invalid_if = 0 ds_invalid_arp = 0 ds_trap_no_if = 0 ds_nowhere_to_go = 0 ds_flow_queue_limit_exceeded = 0 ds_flow_no_memory = 0 ds_flow_invalid_protocol = 0 ds_flow_nat_no_rflow = 0 ds_flow_action_drop = 0 ds_flow_action_invalid = 0 ds_flow_unusable = 0 ds_flow_table_full = 0 ds_interface_tx_discard = 0 ds_interface_drop = 0 ds_duplicated = 0 ds_push = 0 ds_ttl_exceeded = 0 ds_invalid_nh = 0 ds_invalid_label = 0 ds_invalid_protocol = 0 ds_interface_rx_discard = 0 ds_invalid_mcast_source = 0 ds_head_alloc_fail = 0 ds_pcow_fail = 0 ds_mcast_clone_fail = 0 ds_rewrite_fail = 0 ds_misc = 0 ds_invalid_packet = 0 ds_cksum_err = 0 ds_no_fmd = 0 ds_invalid_vnid = 0 ds_frag_err = 0 ds_invalid_source = 0 ds_mcast_df_bit = 0 ds_l2_no_route = 0 ds_vlan_fwd_tx = 0 ds_vlan_fwd_enq = 0 ds_drop_new_flow = 0 ds_flow_evict = 0 ds_trap_original = 0 ds_no_memory = 0 ds_fragment_queue_fail = 0 ds_no_frag_entry = 0 ds_icmp_error = 0 ds_clone_fail = 0 ds_drop_pkts = 0 ] raw_drop_stats= [ ds_discard = 0 ds_pull = 0 ds_invalid_if = 0 ds_invalid_arp = 0 ds_trap_no_if = 0 ds_nowhere_to_go = 0 ds_flow_queue_limit_exceeded = 0 ds_flow_no_memory = 0 ds_flow_invalid_protocol = 0 ds_flow_nat_no_rflow = 0 ds_flow_action_drop = 0 ds_flow_action_invalid = 0 ds_flow_unusable = 0 ds_flow_table_full = 0 ds_interface_tx_discard = 0 ds_interface_drop = 0 ds_duplicated = 0 ds_push = 0 ds_ttl_exceeded = 0 ds_invalid_nh = 0 ds_invalid_label = 0 ds_invalid_protocol = 0 ds_interface_rx_discard = 0 ds_invalid_mcast_source = 0 ds_head_alloc_fail = 0 ds_pcow_fail = 0 ds_mcast_clone_fail = 0 ds_rewrite_fail = 0 ds_misc = 0 ds_invalid_packet = 0 ds_cksum_err = 0 ds_no_fmd = 0 ds_invalid_vnid = 0 ds_frag_err = 0 ds_invalid_source = 0 ds_mcast_df_bit = 0 ds_l2_no_route = 0 ds_vlan_fwd_tx = 0 ds_vlan_fwd_enq = 0 ds_drop_new_flow = 0 ds_flow_evict = 0 ds_trap_original = 0 ds_no_memory = 0 ds_fragment_queue_fail = 0 ds_no_frag_entry = 0 ds_icmp_error = 0 ds_clone_fail = 0 ds_drop_pkts = 0 ] flow_rate= [ added_flows = 0 max_flow_adds_per_second = 0 min_flow_adds_per_second = 0 deleted_flows = 0 max_flow_deletes_per_second = 0 min_flow_deletes_per_second = 0 active_flows = 0 hold_flows = 0 ] raw_phy_if_stats= [ [ _iter262->first = ens6 [ in_pkts = 96 in_bytes = 4312 out_pkts = 211 out_bytes = 9058 ], ] ] raw_phy_if_drop_stats= [ [ _iter263->first = ens6 [ ds_discard = 0 ds_pull = 0 ds_invalid_if = 0 ds_invalid_arp = 0 ds_trap_no_if = 0 ds_nowhere_to_go = 0 ds_flow_queue_limit_exceeded = 0 ds_flow_no_memory = 0 ds_flow_invalid_protocol = 0 ds_flow_nat_no_rflow = 0 ds_flow_action_drop = 0 ds_flow_action_invalid = 0 ds_flow_unusable = 0 ds_flow_table_full = 0 ds_interface_tx_discard = 0 ds_interface_drop = 0 ds_duplicated = 0 ds_push = 0 ds_ttl_exceeded = 0 ds_invalid_nh = 0 ds_invalid_label = 0 ds_invalid_protocol = 0 ds_interface_rx_discard = 0 ds_invalid_mcast_source = 0 ds_head_alloc_fail = 0 ds_pcow_fail = 0 ds_mcast_clone_fail = 0 ds_rewrite_fail = 0 ds_misc = 0 ds_invalid_packet = 0 ds_cksum_err = 0 ds_no_fmd = 0 ds_invalid_vnid = 0 ds_frag_err = 0 ds_invalid_source = 0 ds_mcast_df_bit = 0 ds_l2_no_route = 0 ds_vlan_fwd_tx = 0 ds_vlan_fwd_enq = 0 ds_drop_new_flow = 0 ds_flow_evict = 0 ds_trap_original = 0 ds_no_memory = 0 ds_fragment_queue_fail = 0 ds_no_frag_entry = 0 ds_icmp_error = 0 ds_clone_fail = 0 ds_drop_pkts = 0 ], ] ] ] 2025-09-01 Mon 02:17:54:425.932 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011160806912, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: VrouterStats 2025-09-01 Mon 02:17:54:425.957 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011160806912, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:17:54:426.043 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011160806912, Pid 113882]: SANDESH: Send FAILED: 1756693074425117 [SYS_NOTICE]: VrouterControlStatsTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3993-1 raw_xmpp_stats= [ [ _iter306->first = 10.20.0.14 [ reconnects = 0 in_msgs = 0 out_msgs = 0 ], ] ] raw_ifmap_stats= [ [ _iter307->first = link_delete_parse_errors _iter307->second = 0, _iter307->first = link_update_parse_errors _iter307->second = 0, _iter307->first = node_delete_parse_errors _iter307->second = 0, _iter307->first = node_update_parse_errors _iter307->second = 0, ] ] raw_rt_table_size= [ [ _iter308->first = default-domain:default-project:ip-fabric:__default__ [ inet4_unicast = 4 inet4_multicast = 0 evpn = 0 bridge = 2 inet6_unicast = 2 ], _iter308->first = default-domain:default-project:ip-fabric:ip-fabric [ inet4_unicast = 2 inet4_multicast = 0 evpn = 1 bridge = 2 inet6_unicast = 0 ], ] ] ] 2025-09-01 Mon 02:17:54:426.064 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011160806912, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: VrouterControlStatsTrace 2025-09-01 Mon 02:17:55:445.439 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011156608512, Pid 113882]: Processing scm::EvIdleHoldTimerExpired in state Idle 2025-09-01 Mon 02:17:55:445.761 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011156608512, Pid 113882]: Connect : Start Connect timer 10.0.0.56:8086 2025-09-01 Mon 02:17:55:445.808 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011171994240, Pid 113882]: -(-1) OnSessionEvent TCP Connect Failed 2025-09-01 Mon 02:17:55:445.852 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011171994240, Pid 113882]: -(-1):EvTcpConnectFail 2025-09-01 Mon 02:17:55:445.932 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011156608512, Pid 113882]: Processing scm::EvSandeshSend in state Connect 2025-09-01 Mon 02:17:55:445.980 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011156608512, Pid 113882]: SANDESH: Send FAILED: 1756693075445523 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-3993-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.14 server_addrs= [ [ (*_iter6) = 10.20.0.14:5269, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.14 server_addrs= [ [ (*_iter6) = 10.20.0.14:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.56:8086, ] ] status = Initializing description = Idle : EvIdleHoldTimerExpired -> Connect ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.14, XMPP:dns-server:10.20.0.14, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:17:55:445.997 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011156608512, Pid 113882]: Wrong state: Connect for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:17:55:446.015 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011156608512, Pid 113882]: Processing scm::EvTcpConnectFail in state Connect 2025-09-01 Mon 02:17:55:446.084 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011156608512, Pid 113882]: Connect : EvTcpConnectFail 2025-09-01 Mon 02:17:55:446.191 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011156608512, Pid 113882]: TCP [SYS_ERR]: TcpSessionMessageLog: Session -(-1) > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:17:55:446.242 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011156608512, Pid 113882]: Idle 2025-09-01 Mon 02:17:55:446.372 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011156608512, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:17:55:446.674 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011156608512, Pid 113882]: SANDESH: Send FAILED: 1756693075445894 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3993-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Connect successful_connections = 0 pid = 113882 http_port = 8085 start_time = 1756692234402667 collector_name = collector_ip = 10.0.0.56:8086 collector_list= [ [ (*_iter40) = 10.0.0.56:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ComputeCpuStateTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 14 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 69 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 122 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 190 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterControlStatsTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 28 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 42 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 260 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 66 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 259, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 540, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:17:55:446.716 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011156608512, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:17:55:446.735 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011156608512, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:17:55:446.768 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011156608512, Pid 113882]: SANDESH: Send FAILED: 1756693075446138 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-3993-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.14 server_addrs= [ [ (*_iter6) = 10.20.0.14:5269, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.14 server_addrs= [ [ (*_iter6) = 10.20.0.14:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.14, XMPP:dns-server:10.20.0.14, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:17:55:446.780 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011156608512, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:17:55:446.795 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011156608512, Pid 113882]: Processing scm::EvTcpDeleteSession in state Idle 2025-09-01 Mon 02:17:55:446.843 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011156608512, Pid 113882]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:17:55:447.067 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011156608512, Pid 113882]: SANDESH: Send FAILED: 1756693075446345 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3993-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Idle successful_connections = 0 pid = 113882 http_port = 8085 start_time = 1756692234402667 collector_name = collector_ip = 10.0.0.56:8086 collector_list= [ [ (*_iter40) = 10.0.0.56:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ComputeCpuStateTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 14 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 70 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 122 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 191 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterControlStatsTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 28 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 42 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 260 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 66 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 260, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 541, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:17:55:447.099 UTC cn-jenkins-deploy-platform-ansible-os-3993-1 [Thread 140011156608512, Pid 113882]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace