2025-09-01 Mon 02:02:25:383.223 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848886536832, Pid 111888]: vhost interface name : vhost0 2025-09-01 Mon 02:02:25:383.482 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848886536832, Pid 111888]: vhost IP Address : 10.20.0.17/24 2025-09-01 Mon 02:02:25:383.533 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848886536832, Pid 111888]: vhost gateway : 0.0.0.0 2025-09-01 Mon 02:02:25:383.552 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848886536832, Pid 111888]: Ethernet ports : ens6 2025-09-01 Mon 02:02:25:383.568 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848886536832, Pid 111888]: Loopback IP : 0.0.0.0 2025-09-01 Mon 02:02:25:383.585 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848886536832, Pid 111888]: Xmpp Servers : 10.20.0.17:5269 2025-09-01 Mon 02:02:25:383.604 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848886536832, Pid 111888]: Xmpp Authentication : 0 2025-09-01 Mon 02:02:25:383.619 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848886536832, Pid 111888]: Cluster-Name : 2025-09-01 Mon 02:02:25:383.636 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848886536832, Pid 111888]: DNS Servers : 10.20.0.17:53 2025-09-01 Mon 02:02:25:383.652 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848886536832, Pid 111888]: DNS client port : 8098 2025-09-01 Mon 02:02:25:383.667 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848886536832, Pid 111888]: DNS timeout : 3000 2025-09-01 Mon 02:02:25:383.682 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848886536832, Pid 111888]: DNS max retries : 2 2025-09-01 Mon 02:02:25:383.697 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848886536832, Pid 111888]: Xmpp Dns Authentication : 0 2025-09-01 Mon 02:02:25:383.712 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848886536832, Pid 111888]: TSN Servers : [] 2025-09-01 Mon 02:02:25:383.728 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848886536832, Pid 111888]: COLLECTOR Servers : 10.0.0.254:8086 2025-09-01 Mon 02:02:25:383.743 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848886536832, Pid 111888]: Tunnel-Type : MPLSoGRE 2025-09-01 Mon 02:02:25:383.758 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848886536832, Pid 111888]: Metadata-Proxy Shared Secret: contrail 2025-09-01 Mon 02:02:25:383.773 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848886536832, Pid 111888]: Metadata-Proxy Port : 8097 2025-09-01 Mon 02:02:25:383.788 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848886536832, Pid 111888]: Metadata-Proxy SSL Flag : 0 2025-09-01 Mon 02:02:25:383.803 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848886536832, Pid 111888]: Linklocal Max System Flows : 16384 2025-09-01 Mon 02:02:25:383.818 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848886536832, Pid 111888]: Linklocal Max Vm Flows : 16384 2025-09-01 Mon 02:02:25:383.833 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848886536832, Pid 111888]: Flow cache timeout : 0 2025-09-01 Mon 02:02:25:383.848 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848886536832, Pid 111888]: Stale Interface cleanup timeout : 60 2025-09-01 Mon 02:02:25:383.866 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848886536832, Pid 111888]: Flow thread count : 1 2025-09-01 Mon 02:02:25:383.888 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848886536832, Pid 111888]: Flow latency limit : 0 2025-09-01 Mon 02:02:25:383.910 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848886536832, Pid 111888]: Flow index-mgr sm log count : 0 2025-09-01 Mon 02:02:25:383.934 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848886536832, Pid 111888]: Flow add-tokens : 50 2025-09-01 Mon 02:02:25:383.962 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848886536832, Pid 111888]: Flow ksync-tokens : 25 2025-09-01 Mon 02:02:25:383.977 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848886536832, Pid 111888]: Flow del-tokens : 16 2025-09-01 Mon 02:02:25:383.992 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848886536832, Pid 111888]: Flow update-tokens : 16 2025-09-01 Mon 02:02:25:384.046 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848886536832, Pid 111888]: Pin flow netlink task to CPU: 2025-09-01 Mon 02:02:25:384.063 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848886536832, Pid 111888]: Maximum sessions : 100 2025-09-01 Mon 02:02:25:384.124 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848886536832, Pid 111888]: Maximum session aggregates : 8 2025-09-01 Mon 02:02:25:384.140 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848886536832, Pid 111888]: Maximum session endpoints : 5 2025-09-01 Mon 02:02:25:384.155 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848886536832, Pid 111888]: Fabric SNAT hash table size : 4096 2025-09-01 Mon 02:02:25:384.170 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848886536832, Pid 111888]: Flow excluding Router ID in hash :0 2025-09-01 Mon 02:02:25:384.188 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848886536832, Pid 111888]: Agent Mode : Vrouter 2025-09-01 Mon 02:02:25:384.203 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848886536832, Pid 111888]: Gateway Mode : None 2025-09-01 Mon 02:02:25:384.218 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848886536832, Pid 111888]: DHCP Relay Mode : 0 2025-09-01 Mon 02:02:25:384.233 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848886536832, Pid 111888]: Service instance netns cmd : /usr/local/bin/opencontrail-vrouter-netns 2025-09-01 Mon 02:02:25:384.250 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848886536832, Pid 111888]: Service instance docker cmd : /usr/local/bin/opencontrail-vrouter-docker 2025-09-01 Mon 02:02:25:384.266 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848886536832, Pid 111888]: Service instance workers : 0 2025-09-01 Mon 02:02:25:384.281 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848886536832, Pid 111888]: Service instance timeout : 0 2025-09-01 Mon 02:02:25:384.295 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848886536832, Pid 111888]: Service instance lb ssl : 2025-09-01 Mon 02:02:25:384.310 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848886536832, Pid 111888]: Service instance lbaas auth : 2025-09-01 Mon 02:02:25:384.325 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848886536832, Pid 111888]: Bgp as a service port range : 50000-50512 2025-09-01 Mon 02:02:25:384.339 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848886536832, Pid 111888]: Services queue limit : 1024 2025-09-01 Mon 02:02:25:384.354 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848886536832, Pid 111888]: BGPAAS max shared sessions for service port : 4 2025-09-01 Mon 02:02:25:384.369 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848886536832, Pid 111888]: Sandesh Key file : /etc/contrail/ssl/private/server-privkey.pem 2025-09-01 Mon 02:02:25:384.383 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848886536832, Pid 111888]: Sandesh Cert file : /etc/contrail/ssl/certs/server.pem 2025-09-01 Mon 02:02:25:384.398 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848886536832, Pid 111888]: Sandesh CA Cert : /etc/contrail/ssl/certs/ca-cert.pem 2025-09-01 Mon 02:02:25:384.412 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848886536832, Pid 111888]: Sandesh SSL Enable : 0 2025-09-01 Mon 02:02:25:384.437 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848886536832, Pid 111888]: Introspect SSL Enable : 0 2025-09-01 Mon 02:02:25:384.452 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848886536832, Pid 111888]: Hypervisor mode : kvm 2025-09-01 Mon 02:02:25:384.706 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848886536832, Pid 111888]: Ethernet Port Encap Type : 2025-09-01 Mon 02:02:25:384.729 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848886536832, Pid 111888]: Platform mode : Vrouter on host linux kernel 2025-09-01 Mon 02:02:25:385.187 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: SANDESH: Logging: DISABLED -> ENABLED 2025-09-01 Mon 02:02:25:385.244 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: SANDESH: Logging: LEVEL: [ INVALID ] -> [ SYS_DEBUG ] log4level: [ NOTSET ] -> [ DEBUG ] 2025-09-01 Mon 02:02:25:385.336 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848886536832, Pid 111888]: VRouter [SYS_INFO]: VersionInfoLog: {"build-info":[{"build-version":"v0+nightly","build-time":"2025-09-01 00:13:09.128962","build-user":"root","build-hostname":"jenkins-build-prepare-sandbox-10717-1.novalocal","build-id":"unknown","build-number":"unknown"}]} src/contrail-common/base/misc_utils.cc 41 2025-09-01 Mon 02:02:25:401.402 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Updating dependency tacker rules for virtual-network 2025-09-01 Mon 02:02:25:401.456 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Adding ReactorMap virtual-network : virtual-network-routing-instance -> self 2025-09-01 Mon 02:02:25:401.468 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Adding ReactorMap routing-instance : self -> virtual-network-routing-instance 2025-09-01 Mon 02:02:25:401.476 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Adding ReactorMap routing-instance : virtual-network-routing-instance -> nil 2025-09-01 Mon 02:02:25:401.485 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Updating dependency tacker rules for virtual-network 2025-09-01 Mon 02:02:25:401.491 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Adding ReactorMap virtual-network : virtual-network-access-control-list -> self 2025-09-01 Mon 02:02:25:401.499 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Adding ReactorMap access-control-list : self -> virtual-network-access-control-list 2025-09-01 Mon 02:02:25:401.521 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Adding ReactorMap access-control-list : virtual-network-access-control-list -> nil 2025-09-01 Mon 02:02:25:401.530 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Updating dependency tacker rules for virtual-network 2025-09-01 Mon 02:02:25:401.537 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Adding ReactorMap virtual-network : virtual-network-network-ipam -> self 2025-09-01 Mon 02:02:25:401.544 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Adding ReactorMap virtual-network-network-ipam : self -> virtual-network-network-ipam 2025-09-01 Mon 02:02:25:401.551 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Adding ReactorMap virtual-network-network-ipam : virtual-network-network-ipam -> virtual-network-network-ipam 2025-09-01 Mon 02:02:25:401.558 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Adding ReactorMap network-ipam : virtual-network-network-ipam -> nil 2025-09-01 Mon 02:02:25:401.565 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Updating dependency tacker rules for virtual-network 2025-09-01 Mon 02:02:25:401.579 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Adding ReactorMap virtual-network : virtual-network-qos-config -> self 2025-09-01 Mon 02:02:25:401.588 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Adding ReactorMap qos-config : self -> virtual-network-qos-config 2025-09-01 Mon 02:02:25:401.595 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Adding ReactorMap qos-config : virtual-network-qos-config -> nil 2025-09-01 Mon 02:02:25:401.602 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Updating dependency tacker rules for virtual-network 2025-09-01 Mon 02:02:25:401.608 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Adding ReactorMap virtual-network : virtual-network-security-logging-object -> self 2025-09-01 Mon 02:02:25:401.615 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Adding ReactorMap security-logging-object : self -> virtual-network-security-logging-object 2025-09-01 Mon 02:02:25:401.622 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Adding ReactorMap security-logging-object : virtual-network-security-logging-object -> nil 2025-09-01 Mon 02:02:25:401.629 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Updating dependency tacker rules for virtual-network 2025-09-01 Mon 02:02:25:401.635 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Adding ReactorMap virtual-network : logical-router-virtual-network -> self 2025-09-01 Mon 02:02:25:401.642 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Adding ReactorMap logical-router-virtual-network : self -> logical-router-virtual-network 2025-09-01 Mon 02:02:25:401.670 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Adding ReactorMap logical-router-virtual-network : logical-router-virtual-network -> logical-router-virtual-network 2025-09-01 Mon 02:02:25:401.680 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Adding ReactorMap logical-router : self -> logical-router-virtual-network 2025-09-01 Mon 02:02:25:401.686 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Adding ReactorMap logical-router : logical-router-virtual-network -> nil 2025-09-01 Mon 02:02:25:401.694 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Updating dependency tacker rules for virtual-network 2025-09-01 Mon 02:02:25:401.700 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Adding ReactorMap virtual-network : virtual-network-multicast-policy -> self 2025-09-01 Mon 02:02:25:401.707 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Adding ReactorMap multicast-policy : self -> virtual-network-multicast-policy 2025-09-01 Mon 02:02:25:401.714 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Adding ReactorMap multicast-policy : virtual-network-multicast-policy -> nil 2025-09-01 Mon 02:02:25:401.722 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Updating dependency tacker rules for routing-instance 2025-09-01 Mon 02:02:25:401.739 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Adding ReactorMap routing-instance : virtual-network-routing-instance -> self 2025-09-01 Mon 02:02:25:401.746 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Adding ReactorMap virtual-network : self -> virtual-network-routing-instance 2025-09-01 Mon 02:02:25:401.752 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Adding ReactorMap virtual-network : virtual-network-provider-network -> virtual-network-routing-instance 2025-09-01 Mon 02:02:25:401.763 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Adding ReactorMap virtual-network : self -> virtual-network-provider-network 2025-09-01 Mon 02:02:25:401.769 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Adding ReactorMap virtual-network : virtual-network-provider-network -> nil 2025-09-01 Mon 02:02:25:401.778 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Updating dependency tacker rules for security-group 2025-09-01 Mon 02:02:25:401.784 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Adding ReactorMap security-group : security-group-access-control-list -> self 2025-09-01 Mon 02:02:25:401.791 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Adding ReactorMap access-control-list : self -> security-group-access-control-list 2025-09-01 Mon 02:02:25:401.797 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Adding ReactorMap access-control-list : security-group-access-control-list -> nil 2025-09-01 Mon 02:02:25:401.806 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Updating dependency tacker rules for tag 2025-09-01 Mon 02:02:25:401.812 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Adding ReactorMap tag : application-policy-set-tag -> self 2025-09-01 Mon 02:02:25:401.819 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Adding ReactorMap application-policy-set : self -> application-policy-set-tag 2025-09-01 Mon 02:02:25:401.826 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Adding ReactorMap application-policy-set : policy-management-application-policy-set -> application-policy-set-tag 2025-09-01 Mon 02:02:25:401.833 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Adding ReactorMap policy-management : policy-management-application-policy-set -> nil 2025-09-01 Mon 02:02:25:401.841 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Updating dependency tacker rules for virtual-machine-interface 2025-09-01 Mon 02:02:25:401.847 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Adding ReactorMap virtual-machine-interface : virtual-machine-interface-virtual-network -> self 2025-09-01 Mon 02:02:25:401.854 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Adding ReactorMap virtual-network : self -> virtual-machine-interface-virtual-network 2025-09-01 Mon 02:02:25:401.861 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Adding ReactorMap virtual-network : virtual-network-network-ipam -> virtual-machine-interface-virtual-network 2025-09-01 Mon 02:02:25:401.867 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Adding ReactorMap virtual-network-network-ipam : self -> virtual-network-network-ipam 2025-09-01 Mon 02:02:25:401.874 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Adding ReactorMap virtual-network-network-ipam : virtual-network-network-ipam -> nil 2025-09-01 Mon 02:02:25:401.881 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Updating dependency tacker rules for virtual-machine-interface 2025-09-01 Mon 02:02:25:401.887 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Adding ReactorMap virtual-machine-interface : virtual-machine-interface-virtual-machine -> self 2025-09-01 Mon 02:02:25:401.894 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Adding ReactorMap virtual-machine : self -> virtual-machine-interface-virtual-machine 2025-09-01 Mon 02:02:25:401.904 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Adding ReactorMap virtual-machine : virtual-machine-interface-virtual-machine -> nil 2025-09-01 Mon 02:02:25:401.911 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Updating dependency tacker rules for virtual-machine-interface 2025-09-01 Mon 02:02:25:401.917 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Adding ReactorMap virtual-machine-interface : virtual-machine-interface-sub-interface -> self 2025-09-01 Mon 02:02:25:401.924 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Adding ReactorMap virtual-machine-interface : self -> virtual-machine-interface-sub-interface 2025-09-01 Mon 02:02:25:401.930 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Adding ReactorMap virtual-machine-interface : virtual-machine-interface-sub-interface -> nil 2025-09-01 Mon 02:02:25:401.937 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Updating dependency tacker rules for virtual-machine-interface 2025-09-01 Mon 02:02:25:401.943 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Adding ReactorMap virtual-machine-interface : virtual-machine-interface-virtual-network -> self 2025-09-01 Mon 02:02:25:401.950 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Adding ReactorMap virtual-network : self -> virtual-machine-interface-virtual-network 2025-09-01 Mon 02:02:25:401.956 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Adding ReactorMap virtual-network : virtual-network-provider-network -> virtual-machine-interface-virtual-network 2025-09-01 Mon 02:02:25:401.963 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Adding ReactorMap virtual-network : self -> virtual-network-provider-network 2025-09-01 Mon 02:02:25:401.969 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Adding ReactorMap virtual-network : virtual-network-provider-network -> nil 2025-09-01 Mon 02:02:25:401.977 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Updating dependency tacker rules for virtual-machine-interface 2025-09-01 Mon 02:02:25:401.987 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Adding ReactorMap virtual-machine-interface : virtual-machine-interface-routing-instance -> self 2025-09-01 Mon 02:02:25:401.994 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Adding ReactorMap virtual-machine-interface-routing-instance : self -> virtual-machine-interface-routing-instance 2025-09-01 Mon 02:02:25:402.001 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Adding ReactorMap virtual-machine-interface-routing-instance : virtual-machine-interface-routing-instance -> virtual-machine-interface-routing-instance 2025-09-01 Mon 02:02:25:402.008 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Adding ReactorMap routing-instance : self -> virtual-machine-interface-routing-instance 2025-09-01 Mon 02:02:25:402.015 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Adding ReactorMap routing-instance : virtual-machine-interface-routing-instance -> nil 2025-09-01 Mon 02:02:25:402.022 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Updating dependency tacker rules for virtual-machine-interface 2025-09-01 Mon 02:02:25:402.028 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Adding ReactorMap virtual-machine-interface : virtual-machine-interface-security-group -> self 2025-09-01 Mon 02:02:25:402.035 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Adding ReactorMap security-group : self -> virtual-machine-interface-security-group 2025-09-01 Mon 02:02:25:402.044 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Adding ReactorMap security-group : virtual-machine-interface-security-group -> nil 2025-09-01 Mon 02:02:25:402.052 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Updating dependency tacker rules for virtual-machine-interface 2025-09-01 Mon 02:02:25:402.058 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Adding ReactorMap virtual-machine-interface : alias-ip-virtual-machine-interface -> self 2025-09-01 Mon 02:02:25:402.066 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Adding ReactorMap alias-ip : self -> alias-ip-virtual-machine-interface 2025-09-01 Mon 02:02:25:402.072 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Adding ReactorMap alias-ip : alias-ip-pool-alias-ip -> alias-ip-virtual-machine-interface 2025-09-01 Mon 02:02:25:402.080 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Adding ReactorMap alias-ip-pool : virtual-network-alias-ip-pool -> alias-ip-pool-alias-ip 2025-09-01 Mon 02:02:25:402.086 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Adding ReactorMap virtual-network : self -> virtual-network-alias-ip-pool 2025-09-01 Mon 02:02:25:402.103 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Adding ReactorMap virtual-network : virtual-network-alias-ip-pool -> nil 2025-09-01 Mon 02:02:25:402.112 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Updating dependency tacker rules for virtual-machine-interface 2025-09-01 Mon 02:02:25:402.118 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Adding ReactorMap virtual-machine-interface : floating-ip-virtual-machine-interface -> self 2025-09-01 Mon 02:02:25:402.127 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Adding ReactorMap floating-ip : self -> floating-ip-virtual-machine-interface 2025-09-01 Mon 02:02:25:402.135 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Adding ReactorMap floating-ip : floating-ip-pool-floating-ip -> floating-ip-virtual-machine-interface 2025-09-01 Mon 02:02:25:402.143 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Adding ReactorMap floating-ip-pool : virtual-network-floating-ip-pool -> floating-ip-pool-floating-ip 2025-09-01 Mon 02:02:25:402.150 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Adding ReactorMap virtual-network : self -> virtual-network-floating-ip-pool 2025-09-01 Mon 02:02:25:402.157 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Adding ReactorMap virtual-network : virtual-network-provider-network -> virtual-network-floating-ip-pool 2025-09-01 Mon 02:02:25:402.163 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Adding ReactorMap virtual-network : self -> virtual-network-provider-network 2025-09-01 Mon 02:02:25:402.169 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Adding ReactorMap virtual-network : virtual-network-provider-network -> nil 2025-09-01 Mon 02:02:25:402.177 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Updating dependency tacker rules for virtual-machine-interface 2025-09-01 Mon 02:02:25:402.183 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Adding ReactorMap virtual-machine-interface : floating-ip-virtual-machine-interface -> self 2025-09-01 Mon 02:02:25:402.189 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Adding ReactorMap floating-ip : self -> floating-ip-virtual-machine-interface 2025-09-01 Mon 02:02:25:402.199 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Adding ReactorMap floating-ip : floating-ip-pool-floating-ip -> floating-ip-virtual-machine-interface 2025-09-01 Mon 02:02:25:402.206 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Adding ReactorMap floating-ip-pool : virtual-network-floating-ip-pool -> floating-ip-pool-floating-ip 2025-09-01 Mon 02:02:25:402.212 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Adding ReactorMap virtual-network : self -> virtual-network-floating-ip-pool 2025-09-01 Mon 02:02:25:402.218 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Adding ReactorMap virtual-network : virtual-network-routing-instance -> virtual-network-floating-ip-pool 2025-09-01 Mon 02:02:25:402.225 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Adding ReactorMap routing-instance : self -> virtual-network-routing-instance 2025-09-01 Mon 02:02:25:402.231 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Adding ReactorMap routing-instance : virtual-network-routing-instance -> nil 2025-09-01 Mon 02:02:25:402.238 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Updating dependency tacker rules for virtual-machine-interface 2025-09-01 Mon 02:02:25:402.244 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Adding ReactorMap virtual-machine-interface : floating-ip-virtual-machine-interface -> self 2025-09-01 Mon 02:02:25:402.251 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Adding ReactorMap floating-ip : self -> floating-ip-virtual-machine-interface 2025-09-01 Mon 02:02:25:402.257 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Adding ReactorMap floating-ip : instance-ip-floating-ip -> floating-ip-virtual-machine-interface 2025-09-01 Mon 02:02:25:402.264 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Adding ReactorMap instance-ip : instance-ip-virtual-network -> instance-ip-floating-ip 2025-09-01 Mon 02:02:25:402.270 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Adding ReactorMap virtual-network : self -> instance-ip-virtual-network 2025-09-01 Mon 02:02:25:402.277 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Adding ReactorMap virtual-network : instance-ip-virtual-network -> nil 2025-09-01 Mon 02:02:25:402.284 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Updating dependency tacker rules for virtual-machine-interface 2025-09-01 Mon 02:02:25:402.290 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Adding ReactorMap virtual-machine-interface : instance-ip-virtual-machine-interface -> self 2025-09-01 Mon 02:02:25:402.296 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Adding ReactorMap instance-ip : self -> instance-ip-virtual-machine-interface 2025-09-01 Mon 02:02:25:402.303 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Adding ReactorMap instance-ip : instance-ip-virtual-machine-interface -> nil 2025-09-01 Mon 02:02:25:402.310 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Updating dependency tacker rules for virtual-machine-interface 2025-09-01 Mon 02:02:25:402.316 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Adding ReactorMap virtual-machine-interface : virtual-machine-interface-route-table -> self 2025-09-01 Mon 02:02:25:402.323 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Adding ReactorMap interface-route-table : self -> virtual-machine-interface-route-table 2025-09-01 Mon 02:02:25:402.333 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Adding ReactorMap interface-route-table : virtual-machine-interface-route-table -> nil 2025-09-01 Mon 02:02:25:402.340 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Updating dependency tacker rules for virtual-machine-interface 2025-09-01 Mon 02:02:25:402.346 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Adding ReactorMap virtual-machine-interface : subnet-virtual-machine-interface -> self 2025-09-01 Mon 02:02:25:402.354 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Adding ReactorMap subnet : self -> subnet-virtual-machine-interface 2025-09-01 Mon 02:02:25:402.360 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Adding ReactorMap subnet : subnet-virtual-machine-interface -> nil 2025-09-01 Mon 02:02:25:402.368 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Updating dependency tacker rules for virtual-machine-interface 2025-09-01 Mon 02:02:25:402.375 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Adding ReactorMap virtual-machine-interface : logical-interface-virtual-machine-interface -> self 2025-09-01 Mon 02:02:25:402.382 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Adding ReactorMap logical-interface : physical-interface-logical-interface -> logical-interface-virtual-machine-interface 2025-09-01 Mon 02:02:25:402.390 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Adding ReactorMap physical-interface : physical-router-physical-interface -> physical-interface-logical-interface 2025-09-01 Mon 02:02:25:402.397 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Adding ReactorMap physical-router : self -> physical-router-physical-interface 2025-09-01 Mon 02:02:25:402.403 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Adding ReactorMap physical-router : physical-router-physical-interface -> nil 2025-09-01 Mon 02:02:25:402.411 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Updating dependency tacker rules for virtual-machine-interface 2025-09-01 Mon 02:02:25:402.417 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Adding ReactorMap virtual-machine-interface : virtual-port-group-virtual-machine-interface -> self 2025-09-01 Mon 02:02:25:402.425 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Adding ReactorMap virtual-port-group : virtual-port-group-physical-interface -> virtual-port-group-virtual-machine-interface 2025-09-01 Mon 02:02:25:402.431 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Adding ReactorMap physical-interface : physical-router-physical-interface -> virtual-port-group-physical-interface 2025-09-01 Mon 02:02:25:402.438 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Adding ReactorMap physical-router : self -> physical-router-physical-interface 2025-09-01 Mon 02:02:25:402.444 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Adding ReactorMap physical-router : physical-router-physical-interface -> nil 2025-09-01 Mon 02:02:25:402.452 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Updating dependency tacker rules for virtual-machine-interface 2025-09-01 Mon 02:02:25:402.458 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Adding ReactorMap virtual-machine-interface : bgpaas-virtual-machine-interface -> self 2025-09-01 Mon 02:02:25:402.465 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Adding ReactorMap bgp-as-a-service : self -> bgpaas-virtual-machine-interface 2025-09-01 Mon 02:02:25:402.476 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Adding ReactorMap bgp-as-a-service : bgpaas-bgp-router -> bgpaas-virtual-machine-interface 2025-09-01 Mon 02:02:25:402.484 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Adding ReactorMap bgp-router : self -> bgpaas-bgp-router 2025-09-01 Mon 02:02:25:402.490 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Adding ReactorMap bgp-router : instance-bgp-router -> bgpaas-bgp-router 2025-09-01 Mon 02:02:25:402.497 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Adding ReactorMap routing-instance : self -> instance-bgp-router 2025-09-01 Mon 02:02:25:402.518 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Adding ReactorMap routing-instance : instance-bgp-router -> nil 2025-09-01 Mon 02:02:25:402.526 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Updating dependency tacker rules for virtual-machine-interface 2025-09-01 Mon 02:02:25:402.532 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Adding ReactorMap virtual-machine-interface : bgpaas-virtual-machine-interface -> self 2025-09-01 Mon 02:02:25:402.538 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Adding ReactorMap bgp-as-a-service : self -> bgpaas-virtual-machine-interface 2025-09-01 Mon 02:02:25:402.545 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Adding ReactorMap bgp-as-a-service : bgpaas-health-check -> bgpaas-virtual-machine-interface 2025-09-01 Mon 02:02:25:402.552 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Adding ReactorMap service-health-check : self -> bgpaas-health-check 2025-09-01 Mon 02:02:25:402.559 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Adding ReactorMap service-health-check : bgpaas-health-check -> nil 2025-09-01 Mon 02:02:25:402.566 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Updating dependency tacker rules for virtual-machine-interface 2025-09-01 Mon 02:02:25:402.572 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Adding ReactorMap virtual-machine-interface : virtual-machine-interface-qos-config -> self 2025-09-01 Mon 02:02:25:402.579 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Adding ReactorMap qos-config : self -> virtual-machine-interface-qos-config 2025-09-01 Mon 02:02:25:402.585 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Adding ReactorMap qos-config : virtual-machine-interface-qos-config -> nil 2025-09-01 Mon 02:02:25:402.593 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Updating dependency tacker rules for virtual-machine-interface 2025-09-01 Mon 02:02:25:402.599 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Adding ReactorMap virtual-machine-interface : virtual-machine-interface-security-logging-object -> self 2025-09-01 Mon 02:02:25:402.606 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Adding ReactorMap security-logging-object : self -> virtual-machine-interface-security-logging-object 2025-09-01 Mon 02:02:25:402.613 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Adding ReactorMap security-logging-object : virtual-machine-interface-security-logging-object -> nil 2025-09-01 Mon 02:02:25:402.620 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Updating dependency tacker rules for virtual-machine-interface 2025-09-01 Mon 02:02:25:402.627 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Adding ReactorMap virtual-machine-interface : virtual-machine-interface-bridge-domain -> self 2025-09-01 Mon 02:02:25:402.638 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Adding ReactorMap virtual-machine-interface-bridge-domain : self -> virtual-machine-interface-bridge-domain 2025-09-01 Mon 02:02:25:402.644 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Adding ReactorMap virtual-machine-interface-bridge-domain : virtual-machine-interface-bridge-domain -> virtual-machine-interface-bridge-domain 2025-09-01 Mon 02:02:25:402.651 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Adding ReactorMap bridge-domain : self -> virtual-machine-interface-bridge-domain 2025-09-01 Mon 02:02:25:402.658 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Adding ReactorMap bridge-domain : virtual-network-bridge-domain -> virtual-machine-interface-bridge-domain 2025-09-01 Mon 02:02:25:402.665 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Adding ReactorMap virtual-network : self -> virtual-network-bridge-domain 2025-09-01 Mon 02:02:25:402.672 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Adding ReactorMap virtual-network : virtual-network-bridge-domain -> nil 2025-09-01 Mon 02:02:25:402.679 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Updating dependency tacker rules for virtual-machine-interface 2025-09-01 Mon 02:02:25:402.685 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Adding ReactorMap virtual-machine-interface : virtual-machine-interface-bridge-domain -> self 2025-09-01 Mon 02:02:25:402.692 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Adding ReactorMap virtual-machine-interface-bridge-domain : self -> virtual-machine-interface-bridge-domain 2025-09-01 Mon 02:02:25:402.698 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Adding ReactorMap virtual-machine-interface-bridge-domain : virtual-machine-interface-bridge-domain -> virtual-machine-interface-bridge-domain 2025-09-01 Mon 02:02:25:402.704 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Adding ReactorMap bridge-domain : self -> virtual-machine-interface-bridge-domain 2025-09-01 Mon 02:02:25:402.711 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Adding ReactorMap bridge-domain : virtual-machine-interface-bridge-domain -> nil 2025-09-01 Mon 02:02:25:402.719 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Updating dependency tacker rules for virtual-machine-interface 2025-09-01 Mon 02:02:25:402.725 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Adding ReactorMap virtual-machine-interface : virtual-machine-interface-tag -> self 2025-09-01 Mon 02:02:25:402.731 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Adding ReactorMap tag : self -> virtual-machine-interface-tag 2025-09-01 Mon 02:02:25:402.738 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Adding ReactorMap tag : application-policy-set-tag -> virtual-machine-interface-tag 2025-09-01 Mon 02:02:25:402.745 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Adding ReactorMap application-policy-set : self -> application-policy-set-tag 2025-09-01 Mon 02:02:25:402.751 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Adding ReactorMap application-policy-set : application-policy-set-firewall-policy -> application-policy-set-tag 2025-09-01 Mon 02:02:25:402.758 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Adding ReactorMap application-policy-set-firewall-policy : self -> application-policy-set-firewall-policy 2025-09-01 Mon 02:02:25:402.768 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Adding ReactorMap application-policy-set-firewall-policy : application-policy-set-firewall-policy -> application-policy-set-firewall-policy 2025-09-01 Mon 02:02:25:402.776 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Adding ReactorMap firewall-policy : self -> application-policy-set-firewall-policy 2025-09-01 Mon 02:02:25:402.782 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Adding ReactorMap firewall-policy : application-policy-set-firewall-policy -> nil 2025-09-01 Mon 02:02:25:402.792 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Updating dependency tacker rules for virtual-machine-interface 2025-09-01 Mon 02:02:25:402.799 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Adding ReactorMap virtual-machine-interface : virtual-machine-interface-virtual-network -> self 2025-09-01 Mon 02:02:25:402.805 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Adding ReactorMap virtual-network : self -> virtual-machine-interface-virtual-network 2025-09-01 Mon 02:02:25:402.812 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Adding ReactorMap virtual-network : virtual-network-tag -> virtual-machine-interface-virtual-network 2025-09-01 Mon 02:02:25:402.820 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Adding ReactorMap tag : self -> virtual-network-tag 2025-09-01 Mon 02:02:25:402.830 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Adding ReactorMap tag : application-policy-set-tag -> virtual-network-tag 2025-09-01 Mon 02:02:25:402.840 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Adding ReactorMap application-policy-set : self -> application-policy-set-tag 2025-09-01 Mon 02:02:25:402.850 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Adding ReactorMap application-policy-set : application-policy-set-firewall-policy -> application-policy-set-tag 2025-09-01 Mon 02:02:25:402.859 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Adding ReactorMap application-policy-set-firewall-policy : self -> application-policy-set-firewall-policy 2025-09-01 Mon 02:02:25:402.869 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Adding ReactorMap application-policy-set-firewall-policy : application-policy-set-firewall-policy -> application-policy-set-firewall-policy 2025-09-01 Mon 02:02:25:402.879 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Adding ReactorMap firewall-policy : self -> application-policy-set-firewall-policy 2025-09-01 Mon 02:02:25:402.889 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Adding ReactorMap firewall-policy : application-policy-set-firewall-policy -> nil 2025-09-01 Mon 02:02:25:402.902 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Updating dependency tacker rules for virtual-machine-interface 2025-09-01 Mon 02:02:25:402.909 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Adding ReactorMap virtual-machine-interface : virtual-machine-interface-virtual-machine -> self 2025-09-01 Mon 02:02:25:402.916 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Adding ReactorMap virtual-machine : self -> virtual-machine-interface-virtual-machine 2025-09-01 Mon 02:02:25:402.922 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Adding ReactorMap virtual-machine : virtual-machine-tag -> virtual-machine-interface-virtual-machine 2025-09-01 Mon 02:02:25:402.929 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Adding ReactorMap tag : self -> virtual-machine-tag 2025-09-01 Mon 02:02:25:402.941 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Adding ReactorMap tag : application-policy-set-tag -> virtual-machine-tag 2025-09-01 Mon 02:02:25:402.961 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Adding ReactorMap application-policy-set : self -> application-policy-set-tag 2025-09-01 Mon 02:02:25:402.971 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Adding ReactorMap application-policy-set : application-policy-set-firewall-policy -> application-policy-set-tag 2025-09-01 Mon 02:02:25:402.981 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Adding ReactorMap application-policy-set-firewall-policy : self -> application-policy-set-firewall-policy 2025-09-01 Mon 02:02:25:402.991 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Adding ReactorMap application-policy-set-firewall-policy : application-policy-set-firewall-policy -> application-policy-set-firewall-policy 2025-09-01 Mon 02:02:25:403.000 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Adding ReactorMap firewall-policy : self -> application-policy-set-firewall-policy 2025-09-01 Mon 02:02:25:403.007 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Adding ReactorMap firewall-policy : application-policy-set-firewall-policy -> nil 2025-09-01 Mon 02:02:25:403.016 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Updating dependency tacker rules for virtual-machine-interface 2025-09-01 Mon 02:02:25:403.022 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Adding ReactorMap virtual-machine-interface : virtual-machine-interface-sub-interface -> self 2025-09-01 Mon 02:02:25:403.029 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Adding ReactorMap virtual-machine-interface : self -> virtual-machine-interface-sub-interface 2025-09-01 Mon 02:02:25:403.035 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Adding ReactorMap virtual-machine-interface : virtual-machine-interface-virtual-machine -> virtual-machine-interface-sub-interface 2025-09-01 Mon 02:02:25:403.042 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Adding ReactorMap virtual-machine : self -> virtual-machine-interface-virtual-machine 2025-09-01 Mon 02:02:25:403.048 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Adding ReactorMap virtual-machine : virtual-machine-tag -> virtual-machine-interface-virtual-machine 2025-09-01 Mon 02:02:25:403.054 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Adding ReactorMap tag : self -> virtual-machine-tag 2025-09-01 Mon 02:02:25:403.061 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Adding ReactorMap tag : application-policy-set-tag -> virtual-machine-tag 2025-09-01 Mon 02:02:25:403.067 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Adding ReactorMap application-policy-set : self -> application-policy-set-tag 2025-09-01 Mon 02:02:25:403.073 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Adding ReactorMap application-policy-set : application-policy-set-firewall-policy -> application-policy-set-tag 2025-09-01 Mon 02:02:25:403.079 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Adding ReactorMap application-policy-set-firewall-policy : self -> application-policy-set-firewall-policy 2025-09-01 Mon 02:02:25:403.086 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Adding ReactorMap application-policy-set-firewall-policy : application-policy-set-firewall-policy -> application-policy-set-firewall-policy 2025-09-01 Mon 02:02:25:403.096 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Adding ReactorMap firewall-policy : self -> application-policy-set-firewall-policy 2025-09-01 Mon 02:02:25:403.102 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Adding ReactorMap firewall-policy : application-policy-set-firewall-policy -> nil 2025-09-01 Mon 02:02:25:403.111 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Updating dependency tacker rules for virtual-machine-interface 2025-09-01 Mon 02:02:25:403.117 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Adding ReactorMap virtual-machine-interface : project-virtual-machine-interface -> self 2025-09-01 Mon 02:02:25:403.125 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Adding ReactorMap project : self -> project-virtual-machine-interface 2025-09-01 Mon 02:02:25:403.132 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Adding ReactorMap project : project-tag -> project-virtual-machine-interface 2025-09-01 Mon 02:02:25:403.138 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Adding ReactorMap tag : self -> project-tag 2025-09-01 Mon 02:02:25:403.145 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Adding ReactorMap tag : application-policy-set-tag -> project-tag 2025-09-01 Mon 02:02:25:403.151 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Adding ReactorMap application-policy-set : self -> application-policy-set-tag 2025-09-01 Mon 02:02:25:403.157 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Adding ReactorMap application-policy-set : application-policy-set-firewall-policy -> application-policy-set-tag 2025-09-01 Mon 02:02:25:403.163 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Adding ReactorMap application-policy-set-firewall-policy : self -> application-policy-set-firewall-policy 2025-09-01 Mon 02:02:25:403.169 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Adding ReactorMap application-policy-set-firewall-policy : application-policy-set-firewall-policy -> application-policy-set-firewall-policy 2025-09-01 Mon 02:02:25:403.175 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Adding ReactorMap firewall-policy : self -> application-policy-set-firewall-policy 2025-09-01 Mon 02:02:25:403.181 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Adding ReactorMap firewall-policy : application-policy-set-firewall-policy -> nil 2025-09-01 Mon 02:02:25:403.189 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Updating dependency tacker rules for virtual-machine-interface 2025-09-01 Mon 02:02:25:403.195 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Adding ReactorMap virtual-machine-interface : port-tuple-interface -> self 2025-09-01 Mon 02:02:25:403.202 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Adding ReactorMap port-tuple : self -> port-tuple-interface 2025-09-01 Mon 02:02:25:403.209 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Adding ReactorMap port-tuple : port-tuple-interface -> port-tuple-interface 2025-09-01 Mon 02:02:25:403.215 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Adding ReactorMap virtual-machine-interface : self -> port-tuple-interface 2025-09-01 Mon 02:02:25:403.222 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Adding ReactorMap virtual-machine-interface : port-tuple-interface -> nil 2025-09-01 Mon 02:02:25:403.229 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Updating dependency tacker rules for virtual-machine-interface 2025-09-01 Mon 02:02:25:403.239 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Adding ReactorMap virtual-machine-interface : logical-router-interface -> self 2025-09-01 Mon 02:02:25:403.245 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Adding ReactorMap logical-router : self -> logical-router-interface 2025-09-01 Mon 02:02:25:403.252 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Adding ReactorMap logical-router : logical-router-interface -> nil 2025-09-01 Mon 02:02:25:403.259 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Updating dependency tacker rules for physical-interface 2025-09-01 Mon 02:02:25:403.265 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Adding ReactorMap physical-interface : physical-router-physical-interface -> self 2025-09-01 Mon 02:02:25:403.272 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Adding ReactorMap physical-router : self -> physical-router-physical-interface 2025-09-01 Mon 02:02:25:403.278 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Adding ReactorMap physical-router : physical-router-physical-interface -> nil 2025-09-01 Mon 02:02:25:403.286 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Updating dependency tacker rules for physical-interface 2025-09-01 Mon 02:02:25:403.292 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Adding ReactorMap physical-interface : virtual-port-group-physical-interface -> self 2025-09-01 Mon 02:02:25:403.301 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Adding ReactorMap virtual-port-group-physical-interface : self -> virtual-port-group-physical-interface 2025-09-01 Mon 02:02:25:403.307 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Adding ReactorMap virtual-port-group-physical-interface : virtual-port-group-physical-interface -> virtual-port-group-physical-interface 2025-09-01 Mon 02:02:25:403.313 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Adding ReactorMap virtual-port-group : self -> virtual-port-group-physical-interface 2025-09-01 Mon 02:02:25:403.320 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Adding ReactorMap virtual-port-group : virtual-port-group-physical-interface -> nil 2025-09-01 Mon 02:02:25:403.327 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Updating dependency tacker rules for logical-interface 2025-09-01 Mon 02:02:25:403.333 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Adding ReactorMap logical-interface : physical-interface-logical-interface -> self 2025-09-01 Mon 02:02:25:403.340 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Adding ReactorMap physical-interface : self -> physical-interface-logical-interface 2025-09-01 Mon 02:02:25:403.346 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Adding ReactorMap physical-interface : physical-router-physical-interface -> physical-interface-logical-interface 2025-09-01 Mon 02:02:25:403.352 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Adding ReactorMap physical-router : self -> physical-router-physical-interface 2025-09-01 Mon 02:02:25:403.358 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Adding ReactorMap physical-router : physical-router-physical-interface -> nil 2025-09-01 Mon 02:02:25:403.365 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Updating dependency tacker rules for logical-interface 2025-09-01 Mon 02:02:25:403.375 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Adding ReactorMap logical-interface : logical-interface-virtual-machine-interface -> self 2025-09-01 Mon 02:02:25:403.382 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Adding ReactorMap virtual-machine-interface : self -> logical-interface-virtual-machine-interface 2025-09-01 Mon 02:02:25:403.388 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Adding ReactorMap virtual-machine-interface : logical-interface-virtual-machine-interface -> nil 2025-09-01 Mon 02:02:25:403.395 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Updating dependency tacker rules for logical-interface 2025-09-01 Mon 02:02:25:403.401 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Adding ReactorMap logical-interface : physical-router-logical-interface -> self 2025-09-01 Mon 02:02:25:403.414 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Adding ReactorMap physical-router : self -> physical-router-logical-interface 2025-09-01 Mon 02:02:25:403.421 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Adding ReactorMap physical-router : physical-router-logical-interface -> nil 2025-09-01 Mon 02:02:25:403.428 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Updating dependency tacker rules for physical-router 2025-09-01 Mon 02:02:25:403.434 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Adding ReactorMap physical-router : physical-router-physical-interface -> self 2025-09-01 Mon 02:02:25:403.441 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Adding ReactorMap physical-interface : self -> physical-router-physical-interface 2025-09-01 Mon 02:02:25:403.447 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Adding ReactorMap physical-interface : physical-router-physical-interface -> nil 2025-09-01 Mon 02:02:25:403.455 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Updating dependency tacker rules for service-health-check 2025-09-01 Mon 02:02:25:403.461 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Adding ReactorMap service-health-check : service-port-health-check -> self 2025-09-01 Mon 02:02:25:403.467 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Adding ReactorMap virtual-machine-interface : self -> service-port-health-check 2025-09-01 Mon 02:02:25:403.474 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Adding ReactorMap virtual-machine-interface : virtual-machine-interface-virtual-network -> service-port-health-check 2025-09-01 Mon 02:02:25:403.481 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Adding ReactorMap virtual-network : virtual-network-network-ipam -> virtual-machine-interface-virtual-network 2025-09-01 Mon 02:02:25:403.487 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Adding ReactorMap virtual-network-network-ipam : self -> virtual-network-network-ipam 2025-09-01 Mon 02:02:25:403.494 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Adding ReactorMap virtual-network-network-ipam : virtual-network-network-ipam -> nil 2025-09-01 Mon 02:02:25:403.512 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Updating dependency tacker rules for service-health-check 2025-09-01 Mon 02:02:25:403.519 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Adding ReactorMap service-health-check : service-port-health-check -> self 2025-09-01 Mon 02:02:25:403.529 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Adding ReactorMap virtual-machine-interface : self -> service-port-health-check 2025-09-01 Mon 02:02:25:403.542 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Adding ReactorMap virtual-machine-interface : port-tuple-interface -> service-port-health-check 2025-09-01 Mon 02:02:25:403.548 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Adding ReactorMap port-tuple : self -> port-tuple-interface 2025-09-01 Mon 02:02:25:403.555 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Adding ReactorMap port-tuple : port-tuple-interface -> port-tuple-interface 2025-09-01 Mon 02:02:25:403.561 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Adding ReactorMap virtual-machine-interface : self -> port-tuple-interface 2025-09-01 Mon 02:02:25:403.567 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Adding ReactorMap virtual-machine-interface : virtual-machine-interface-virtual-network -> port-tuple-interface 2025-09-01 Mon 02:02:25:403.574 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Adding ReactorMap virtual-network : virtual-network-network-ipam -> virtual-machine-interface-virtual-network 2025-09-01 Mon 02:02:25:403.580 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Adding ReactorMap virtual-network-network-ipam : self -> virtual-network-network-ipam 2025-09-01 Mon 02:02:25:403.586 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Adding ReactorMap virtual-network-network-ipam : virtual-network-network-ipam -> nil 2025-09-01 Mon 02:02:25:403.594 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Updating dependency tacker rules for qos-config 2025-09-01 Mon 02:02:25:403.600 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Adding ReactorMap qos-config : global-qos-config-qos-config -> self 2025-09-01 Mon 02:02:25:403.608 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Adding ReactorMap global-qos-config : global-qos-config-qos-config -> nil 2025-09-01 Mon 02:02:25:403.616 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Updating dependency tacker rules for forwarding-class 2025-09-01 Mon 02:02:25:403.622 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Adding ReactorMap forwarding-class : forwarding-class-qos-queue -> self 2025-09-01 Mon 02:02:25:403.629 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Adding ReactorMap qos-queue : self -> forwarding-class-qos-queue 2025-09-01 Mon 02:02:25:403.636 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Adding ReactorMap qos-queue : forwarding-class-qos-queue -> nil 2025-09-01 Mon 02:02:25:403.643 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Updating dependency tacker rules for security-logging-object 2025-09-01 Mon 02:02:25:403.649 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Adding ReactorMap security-logging-object : security-logging-object-network-policy -> self 2025-09-01 Mon 02:02:25:403.656 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Adding ReactorMap network-policy : self -> security-logging-object-network-policy 2025-09-01 Mon 02:02:25:403.663 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Adding ReactorMap network-policy : security-logging-object-network-policy -> nil 2025-09-01 Mon 02:02:25:403.670 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Updating dependency tacker rules for security-logging-object 2025-09-01 Mon 02:02:25:403.680 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Adding ReactorMap security-logging-object : security-logging-object-security-group -> self 2025-09-01 Mon 02:02:25:403.686 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Adding ReactorMap security-group : self -> security-logging-object-security-group 2025-09-01 Mon 02:02:25:403.693 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Adding ReactorMap security-group : security-logging-object-security-group -> nil 2025-09-01 Mon 02:02:25:403.700 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Updating dependency tacker rules for security-logging-object 2025-09-01 Mon 02:02:25:403.706 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Adding ReactorMap security-logging-object : firewall-policy-security-logging-object -> self 2025-09-01 Mon 02:02:25:403.713 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Adding ReactorMap firewall-policy-security-logging-object : self -> firewall-policy-security-logging-object 2025-09-01 Mon 02:02:25:403.720 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Adding ReactorMap firewall-policy-security-logging-object : firewall-policy-security-logging-object -> firewall-policy-security-logging-object 2025-09-01 Mon 02:02:25:403.726 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Adding ReactorMap firewall-policy : firewall-policy-security-logging-object -> nil 2025-09-01 Mon 02:02:25:403.734 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Updating dependency tacker rules for security-logging-object 2025-09-01 Mon 02:02:25:403.740 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Adding ReactorMap security-logging-object : firewall-rule-security-logging-object -> self 2025-09-01 Mon 02:02:25:403.747 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Adding ReactorMap firewall-rule-security-logging-object : self -> firewall-rule-security-logging-object 2025-09-01 Mon 02:02:25:403.753 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Adding ReactorMap firewall-rule-security-logging-object : firewall-rule-security-logging-object -> firewall-rule-security-logging-object 2025-09-01 Mon 02:02:25:403.761 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Adding ReactorMap firewall-rule : firewall-rule-security-logging-object -> nil 2025-09-01 Mon 02:02:25:403.769 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Updating dependency tacker rules for virtual-router 2025-09-01 Mon 02:02:25:403.776 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Adding ReactorMap virtual-router : virtual-router-sub-cluster -> self 2025-09-01 Mon 02:02:25:403.783 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Adding ReactorMap sub-cluster : self -> virtual-router-sub-cluster 2025-09-01 Mon 02:02:25:403.789 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Adding ReactorMap sub-cluster : bgp-router-sub-cluster -> virtual-router-sub-cluster 2025-09-01 Mon 02:02:25:403.795 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Adding ReactorMap bgp-router : self -> bgp-router-sub-cluster 2025-09-01 Mon 02:02:25:403.803 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Adding ReactorMap bgp-router : bgp-router-sub-cluster -> nil 2025-09-01 Mon 02:02:25:403.812 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Updating dependency tacker rules for bridge-domain 2025-09-01 Mon 02:02:25:403.819 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Adding ReactorMap bridge-domain : virtual-network-bridge-domain -> self 2025-09-01 Mon 02:02:25:403.829 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Adding ReactorMap virtual-network : self -> virtual-network-bridge-domain 2025-09-01 Mon 02:02:25:403.836 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Adding ReactorMap virtual-network : virtual-network-routing-instance -> virtual-network-bridge-domain 2025-09-01 Mon 02:02:25:403.842 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Adding ReactorMap routing-instance : self -> virtual-network-routing-instance 2025-09-01 Mon 02:02:25:403.848 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Adding ReactorMap routing-instance : virtual-network-routing-instance -> nil 2025-09-01 Mon 02:02:25:403.855 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Updating dependency tacker rules for bridge-domain 2025-09-01 Mon 02:02:25:403.861 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Adding ReactorMap bridge-domain : virtual-network-bridge-domain -> self 2025-09-01 Mon 02:02:25:403.867 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Adding ReactorMap virtual-network : self -> virtual-network-bridge-domain 2025-09-01 Mon 02:02:25:403.874 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Adding ReactorMap virtual-network : virtual-network-bridge-domain -> nil 2025-09-01 Mon 02:02:25:403.881 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Updating dependency tacker rules for application-policy-set 2025-09-01 Mon 02:02:25:403.887 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Adding ReactorMap application-policy-set : application-policy-set-firewall-policy -> self 2025-09-01 Mon 02:02:25:403.894 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Adding ReactorMap application-policy-set-firewall-policy : self -> application-policy-set-firewall-policy 2025-09-01 Mon 02:02:25:403.900 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Adding ReactorMap application-policy-set-firewall-policy : application-policy-set-firewall-policy -> application-policy-set-firewall-policy 2025-09-01 Mon 02:02:25:403.906 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Adding ReactorMap firewall-policy : self -> application-policy-set-firewall-policy 2025-09-01 Mon 02:02:25:403.913 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Adding ReactorMap firewall-policy : application-policy-set-firewall-policy -> nil 2025-09-01 Mon 02:02:25:403.921 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Updating dependency tacker rules for firewall-policy 2025-09-01 Mon 02:02:25:403.927 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Adding ReactorMap firewall-policy : firewall-policy-firewall-rule -> self 2025-09-01 Mon 02:02:25:403.935 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Adding ReactorMap firewall-policy-firewall-rule : self -> firewall-policy-firewall-rule 2025-09-01 Mon 02:02:25:403.941 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Adding ReactorMap firewall-policy-firewall-rule : firewall-policy-firewall-rule -> firewall-policy-firewall-rule 2025-09-01 Mon 02:02:25:403.947 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Adding ReactorMap firewall-rule : self -> firewall-policy-firewall-rule 2025-09-01 Mon 02:02:25:403.954 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Adding ReactorMap firewall-rule : firewall-rule-service-group -> firewall-policy-firewall-rule 2025-09-01 Mon 02:02:25:403.964 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Adding ReactorMap service-group : self -> firewall-rule-service-group 2025-09-01 Mon 02:02:25:403.971 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Adding ReactorMap service-group : firewall-rule-service-group -> nil 2025-09-01 Mon 02:02:25:403.979 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Updating dependency tacker rules for firewall-policy 2025-09-01 Mon 02:02:25:403.985 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Adding ReactorMap firewall-policy : firewall-policy-firewall-rule -> self 2025-09-01 Mon 02:02:25:403.991 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Adding ReactorMap firewall-policy-firewall-rule : self -> firewall-policy-firewall-rule 2025-09-01 Mon 02:02:25:403.997 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Adding ReactorMap firewall-policy-firewall-rule : firewall-policy-firewall-rule -> firewall-policy-firewall-rule 2025-09-01 Mon 02:02:25:404.004 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Adding ReactorMap firewall-rule : self -> firewall-policy-firewall-rule 2025-09-01 Mon 02:02:25:404.010 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Adding ReactorMap firewall-rule : firewall-rule-address-group -> firewall-policy-firewall-rule 2025-09-01 Mon 02:02:25:404.017 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Adding ReactorMap address-group : self -> firewall-rule-address-group 2025-09-01 Mon 02:02:25:404.024 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Adding ReactorMap address-group : address-group-tag -> firewall-rule-address-group 2025-09-01 Mon 02:02:25:404.030 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Adding ReactorMap tag : self -> address-group-tag 2025-09-01 Mon 02:02:25:404.037 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Adding ReactorMap tag : address-group-tag -> nil 2025-09-01 Mon 02:02:25:404.044 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Updating dependency tacker rules for firewall-policy 2025-09-01 Mon 02:02:25:404.050 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Adding ReactorMap firewall-policy : firewall-policy-firewall-rule -> self 2025-09-01 Mon 02:02:25:404.056 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Adding ReactorMap firewall-policy-firewall-rule : self -> firewall-policy-firewall-rule 2025-09-01 Mon 02:02:25:404.062 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Adding ReactorMap firewall-policy-firewall-rule : firewall-policy-firewall-rule -> firewall-policy-firewall-rule 2025-09-01 Mon 02:02:25:404.068 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Adding ReactorMap firewall-rule : self -> firewall-policy-firewall-rule 2025-09-01 Mon 02:02:25:404.075 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Adding ReactorMap firewall-rule : firewall-rule-tag -> firewall-policy-firewall-rule 2025-09-01 Mon 02:02:25:404.082 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Adding ReactorMap tag : self -> firewall-rule-tag 2025-09-01 Mon 02:02:25:404.088 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Adding ReactorMap tag : firewall-rule-tag -> nil 2025-09-01 Mon 02:02:25:404.095 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Updating dependency tacker rules for firewall-policy 2025-09-01 Mon 02:02:25:404.105 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Adding ReactorMap firewall-policy : firewall-policy-firewall-rule -> self 2025-09-01 Mon 02:02:25:404.111 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Adding ReactorMap firewall-policy-firewall-rule : self -> firewall-policy-firewall-rule 2025-09-01 Mon 02:02:25:404.117 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Adding ReactorMap firewall-policy-firewall-rule : firewall-policy-firewall-rule -> firewall-policy-firewall-rule 2025-09-01 Mon 02:02:25:404.123 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Adding ReactorMap firewall-rule : self -> firewall-policy-firewall-rule 2025-09-01 Mon 02:02:25:404.130 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Adding ReactorMap firewall-rule : firewall-policy-firewall-rule -> nil 2025-09-01 Mon 02:02:25:595.218 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: TCP [SYS_DEBUG]: TcpServerMessageLog: Server 10.20.0.17:8097 Initialization complete src/contrail-common/io/tcp_server.cc 137 2025-09-01 Mon 02:02:25:602.461 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Current receive sock buffer size is 262144 2025-09-01 Mon 02:02:25:602.762 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848850159104, Pid 111888]: KsyncTxQueue CPU pinning policy <>. KsyncTxQueuen not pinned to CPU 2025-09-01 Mon 02:02:25:605.875 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Vrouter family is 34 2025-09-01 Mon 02:02:25:606.956 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Flow table size : 20185088 2025-09-01 Mon 02:02:25:607.072 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Flow table size : 161218560 2025-09-01 Mon 02:02:25:607.093 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: SetHugePages: Bridge table size:20185088 Flow table size:161218560 2025-09-01 Mon 02:02:25:607.128 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Mem mapping hugepage file:/dev/hugepages/bridge size:20185088 2025-09-01 Mon 02:02:25:607.156 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Mem mapped hugepage file:/dev/hugepages/bridge to addr:0x7f3117800000 2025-09-01 Mon 02:02:25:607.172 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Mem mapping hugepage file:/dev/hugepages/flow size:161218560 2025-09-01 Mon 02:02:25:607.183 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Mem mapped hugepage file:/dev/hugepages/flow to addr:0x7f310de00000 2025-09-01 Mon 02:02:25:607.201 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Sending Huge Page configuration to VROUTER 2025-09-01 Mon 02:02:25:662.287 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Huge pages set successfully 2025-09-01 Mon 02:02:25:675.160 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Flow table size : 161218560 2025-09-01 Mon 02:02:25:675.276 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Flow table size : 20185088 2025-09-01 Mon 02:02:25:741.461 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: SANDESH: ROLE : Generator 2025-09-01 Mon 02:02:25:741.492 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: SANDESH: MODULE : contrail-vrouter-agent 2025-09-01 Mon 02:02:25:741.498 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: SANDESH: SOURCE : cn-jenkins-deploy-platform-ansible-os-3992-1 2025-09-01 Mon 02:02:25:741.526 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: SANDESH: NODE TYPE : Compute 2025-09-01 Mon 02:02:25:741.532 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: SANDESH: INSTANCE ID : 0 2025-09-01 Mon 02:02:25:741.537 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: SANDESH: HTTP SERVER PORT : 8085 2025-09-01 Mon 02:02:25:741.543 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: SANDESH: System Log Send Rate Limit: 0 -> 0 2025-09-01 Mon 02:02:25:742.096 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: HTTP Introspect Init 2025-09-01 Mon 02:02:25:742.301 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: TCP [SYS_DEBUG]: TcpServerMessageLog: Server 0.0.0.0:8085 Initialization complete src/contrail-common/io/tcp_server.cc 137 2025-09-01 Mon 02:02:25:742.400 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Sandesh Http Server Port: 8085 2025-09-01 Mon 02:02:25:742.472 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: SANDESH: NOT Writing http_port 8085TO : /tmp/contrail-vrouter-agent.110926.http_port 2025-09-01 Mon 02:02:25:742.483 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: SANDESH: CONNECT TO COLLECTOR: 1 2025-09-01 Mon 02:02:25:743.102 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Idle 2025-09-01 Mon 02:02:25:746.251 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: SANDESH: No Client: 1756692145745381 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3992-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Idle successful_connections = 0 pid = 111888 http_port = 8085 start_time = 1756692145743123 collector_name = collector_ip = 0.0.0.0:0 collector_list= [ [ ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 0, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 1, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 0, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:02:25:746.407 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848854357504, Pid 111888]: Processing scm::EvStart in state Idle 2025-09-01 Mon 02:02:25:746.624 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848854357504, Pid 111888]: Disconnect 2025-09-01 Mon 02:02:25:746.751 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848854357504, Pid 111888]: Processing scm::EvCollectorUpdate in state Disconnect 2025-09-01 Mon 02:02:25:747.156 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848854357504, Pid 111888]: Connect : Start Connect timer 10.0.0.254:8086 2025-09-01 Mon 02:02:25:747.214 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848886536832, Pid 111888]: -(-1) OnSessionEvent TCP Connect Failed 2025-09-01 Mon 02:02:25:747.282 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848886536832, Pid 111888]: -(-1):EvTcpConnectFail 2025-09-01 Mon 02:02:25:747.230 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848854357504, Pid 111888]: Processing scm::EvSandeshSend in state Connect 2025-09-01 Mon 02:02:25:747.387 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848854357504, Pid 111888]: SANDESH: Send FAILED: 1756692145746609 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-3992-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Idle : EvStart -> Disconnect ], ] ] flag_infos= [ [ ] ] description = Collector connection down, No Configuration for self Number of connections:1, Expected: 3 ], ] ] ] 2025-09-01 Mon 02:02:25:747.412 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848854357504, Pid 111888]: Wrong state: Connect for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:02:25:747.442 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848854357504, Pid 111888]: Processing scm::EvSandeshSend in state Connect 2025-09-01 Mon 02:02:25:747.736 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848854357504, Pid 111888]: SANDESH: Send FAILED: 1756692145746731 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3992-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Disconnect successful_connections = 0 pid = 111888 http_port = 8085 start_time = 1756692145743123 collector_name = collector_ip = 0.0.0.0:0 collector_list= [ [ ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 1, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 0, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:02:25:747.816 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848854357504, Pid 111888]: Wrong state: Connect for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:02:25:747.864 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848854357504, Pid 111888]: Processing scm::EvSandeshSend in state Connect 2025-09-01 Mon 02:02:25:748.068 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848854357504, Pid 111888]: SANDESH: Send FAILED: 1756692145746817 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3992-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Disconnect successful_connections = 0 pid = 111888 http_port = 8085 start_time = 1756692145743123 collector_name = collector_ip = 10.0.0.254:8086 collector_list= [ [ (*_iter40) = 10.0.0.254:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 1, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 0, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:02:25:748.110 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848854357504, Pid 111888]: Wrong state: Connect for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:02:25:748.138 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848854357504, Pid 111888]: Processing scm::EvSandeshSend in state Connect 2025-09-01 Mon 02:02:25:748.180 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848854357504, Pid 111888]: SANDESH: Send FAILED: 1756692145746849 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-3992-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.254:8086, ] ] status = Initializing description = Disconnect : EvCollectorUpdate -> Connect ], ] ] flag_infos= [ [ ] ] description = Collector connection down, No Configuration for self Number of connections:1, Expected: 3 ], ] ] ] 2025-09-01 Mon 02:02:25:748.199 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848854357504, Pid 111888]: Wrong state: Connect for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:02:25:748.221 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848854357504, Pid 111888]: Processing scm::EvSandeshSend in state Connect 2025-09-01 Mon 02:02:25:748.428 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848854357504, Pid 111888]: SANDESH: Send FAILED: 1756692145747214 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3992-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Connect successful_connections = 0 pid = 111888 http_port = 8085 start_time = 1756692145743123 collector_name = collector_ip = 10.0.0.254:8086 collector_list= [ [ (*_iter40) = 10.0.0.254:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 1, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 0, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:02:25:748.472 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848854357504, Pid 111888]: Wrong state: Connect for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:02:25:748.519 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848854357504, Pid 111888]: Processing scm::EvTcpConnectFail in state Connect 2025-09-01 Mon 02:02:25:748.541 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848854357504, Pid 111888]: Connect : EvTcpConnectFail 2025-09-01 Mon 02:02:25:748.685 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848854357504, Pid 111888]: TCP [SYS_ERR]: TcpSessionMessageLog: Session -(-1) > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:02:25:748.786 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848854357504, Pid 111888]: Idle 2025-09-01 Mon 02:02:25:749.019 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848854357504, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:25:749.401 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Handle netlink interface message RTM_NEWLINK for interface lo flags 65609 2025-09-01 Mon 02:02:25:749.485 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Handle netlink interface message RTM_NEWLINK for interface ens3 flags 69699 2025-09-01 Mon 02:02:25:749.535 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Handle netlink interface message RTM_NEWLINK for interface ens6 flags 69699 2025-09-01 Mon 02:02:25:749.542 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Handle netlink interface message RTM_NEWLINK for interface docker0 flags 4099 2025-09-01 Mon 02:02:25:749.568 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Handle netlink interface message RTM_NEWLINK for interface pkt1 flags 65601 2025-09-01 Mon 02:02:25:749.574 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Handle netlink interface message RTM_NEWLINK for interface pkt3 flags 65601 2025-09-01 Mon 02:02:25:749.579 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Handle netlink interface message RTM_NEWLINK for interface pkt2 flags 65601 2025-09-01 Mon 02:02:25:749.596 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Handle netlink interface message RTM_NEWLINK for interface vhost0 flags 69699 2025-09-01 Mon 02:02:25:749.602 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Handle netlink interface message RTM_NEWLINK for interface pkt0 flags 69699 2025-09-01 Mon 02:02:25:749.641 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Handle netlink address message RTM_NEWADDR for interface lo 2025-09-01 Mon 02:02:25:749.684 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Handle netlink address message RTM_NEWADDR for interface ens3 2025-09-01 Mon 02:02:25:749.699 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Handle netlink address message RTM_NEWADDR for interface docker0 2025-09-01 Mon 02:02:25:749.712 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Handle netlink address message RTM_NEWADDR for interface vhost0 2025-09-01 Mon 02:02:25:749.739 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Handle netlink address message RTM_NEWADDR for interface lo 2025-09-01 Mon 02:02:25:749.753 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Handle netlink address message RTM_NEWADDR for interface ens3 2025-09-01 Mon 02:02:25:749.776 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Handle netlink address message RTM_NEWADDR for interface vhost0 2025-09-01 Mon 02:02:25:749.790 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Handle netlink address message RTM_NEWADDR for interface pkt0 2025-09-01 Mon 02:02:25:749.849 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Ignoring Netlink route with family 2 table 254 type 1 scope 0 2025-09-01 Mon 02:02:25:749.866 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Handle netlink route message RTM_NEWROUTE : 10.0.0.0/24 Interface ens3 GW 0.0.0.0 2025-09-01 Mon 02:02:25:749.877 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Handle netlink route message RTM_NEWROUTE : 10.0.0.1/32 Interface ens3 GW 0.0.0.0 2025-09-01 Mon 02:02:25:749.888 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Handle netlink route message RTM_NEWROUTE : 10.0.0.2/32 Interface ens3 GW 0.0.0.0 2025-09-01 Mon 02:02:25:749.899 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Handle netlink route message RTM_NEWROUTE : 10.0.0.4/32 Interface ens3 GW 0.0.0.0 2025-09-01 Mon 02:02:25:749.910 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Handle netlink route message RTM_NEWROUTE : 10.20.0.0/24 Interface vhost0 GW 0.0.0.0 2025-09-01 Mon 02:02:25:749.921 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Handle netlink route message RTM_NEWROUTE : 10.20.0.0/24 Interface vhost0 GW 0.0.0.0 2025-09-01 Mon 02:02:25:749.927 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Ignoring Netlink route with family 2 table 254 type 1 scope 0 2025-09-01 Mon 02:02:25:749.939 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Handle netlink route message RTM_NEWROUTE : 172.17.0.0/16 Interface docker0 GW 0.0.0.0 2025-09-01 Mon 02:02:25:749.944 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Ignoring Netlink route with family 2 table 255 type 2 scope 254 2025-09-01 Mon 02:02:25:749.949 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Ignoring Netlink route with family 2 table 255 type 3 scope 253 2025-09-01 Mon 02:02:25:749.955 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Ignoring Netlink route with family 2 table 255 type 2 scope 254 2025-09-01 Mon 02:02:25:749.960 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Ignoring Netlink route with family 2 table 255 type 3 scope 253 2025-09-01 Mon 02:02:25:749.965 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Ignoring Netlink route with family 2 table 255 type 2 scope 254 2025-09-01 Mon 02:02:25:749.970 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Ignoring Netlink route with family 2 table 255 type 2 scope 254 2025-09-01 Mon 02:02:25:749.975 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Ignoring Netlink route with family 2 table 255 type 3 scope 253 2025-09-01 Mon 02:02:25:749.980 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Ignoring Netlink route with family 2 table 255 type 2 scope 254 2025-09-01 Mon 02:02:25:749.985 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Ignoring Netlink route with family 2 table 255 type 3 scope 253 2025-09-01 Mon 02:02:25:750.002 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Ignoring Netlink route with family 10 table 254 type 1 scope 0 2025-09-01 Mon 02:02:25:750.012 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Ignoring Netlink route with family 10 table 254 type 1 scope 0 2025-09-01 Mon 02:02:25:750.017 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Ignoring Netlink route with family 10 table 254 type 1 scope 0 2025-09-01 Mon 02:02:25:750.022 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Ignoring Netlink route with family 10 table 254 type 1 scope 0 2025-09-01 Mon 02:02:25:750.027 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Ignoring Netlink route with family 10 table 255 type 2 scope 0 2025-09-01 Mon 02:02:25:750.032 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Ignoring Netlink route with family 10 table 255 type 2 scope 0 2025-09-01 Mon 02:02:25:750.038 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Ignoring Netlink route with family 10 table 255 type 2 scope 0 2025-09-01 Mon 02:02:25:750.043 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Ignoring Netlink route with family 10 table 255 type 5 scope 0 2025-09-01 Mon 02:02:25:750.048 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Ignoring Netlink route with family 10 table 255 type 5 scope 0 2025-09-01 Mon 02:02:25:750.053 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Ignoring Netlink route with family 10 table 255 type 5 scope 0 2025-09-01 Mon 02:02:25:750.058 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Ignoring Netlink route with family 10 table 255 type 5 scope 0 2025-09-01 Mon 02:02:25:751.623 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848854357504, Pid 111888]: SANDESH: Send FAILED: 1756692145747480 AgentVrf [SYS_INFO]: VrfObjectLog: vrf= [ event = Addition name = default-domain:default-project:ip-fabric:__default__ index = 0 ] file = controller/src/vnsw/agent/oper/vrf.cc line = 414 2025-09-01 Mon 02:02:25:751.648 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848854357504, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: VrfObjectLog 2025-09-01 Mon 02:02:25:751.662 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848854357504, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:25:751.676 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848854357504, Pid 111888]: SANDESH: Send FAILED: 1756692145747532 AgentVrf [SYS_INFO]: VrfObjectLog: vrf= [ event = Addition name = default-domain:default-project:ip-fabric:ip-fabric index = 1 ] file = controller/src/vnsw/agent/oper/vrf.cc line = 414 2025-09-01 Mon 02:02:25:751.683 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848854357504, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: VrfObjectLog 2025-09-01 Mon 02:02:25:751.691 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848854357504, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:25:751.706 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848854357504, Pid 111888]: SANDESH: Send FAILED: 1756692145748606 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-3992-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = Collector connection down, No Configuration for self Number of connections:1, Expected: 3 ], ] ] ] 2025-09-01 Mon 02:02:25:751.712 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848854357504, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:02:25:751.720 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848854357504, Pid 111888]: Processing scm::EvTcpDeleteSession in state Idle 2025-09-01 Mon 02:02:25:751.784 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848854357504, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:25:751.896 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848854357504, Pid 111888]: SANDESH: Send FAILED: 1756692145748950 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3992-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Idle successful_connections = 0 pid = 111888 http_port = 8085 start_time = 1756692145743123 collector_name = collector_ip = 10.0.0.254:8086 collector_list= [ [ (*_iter40) = 10.0.0.254:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 3 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 2, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 5, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:02:25:751.913 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848854357504, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:02:25:769.420 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: TCP [SYS_DEBUG]: TcpServerMessageLog: Server 0.0.0.0:9091 Initialization complete src/contrail-common/io/tcp_server.cc 137 2025-09-01 Mon 02:02:25:769.589 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Router ID configured. Connection to controller initiated 2025-09-01 Mon 02:02:25:774.381 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848854357504, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:25:774.450 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848854357504, Pid 111888]: SANDESH: Send FAILED: 1756692145774259 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.17 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:25:774.470 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848854357504, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:25:774.494 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848854357504, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:25:774.546 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848854357504, Pid 111888]: SANDESH: Send FAILED: 1756692145774396 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-3992-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.17 server_addrs= [ [ (*_iter6) = 10.20.0.17:5269, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.17, Collector connection down, No Configuration for self Number of connections:2, Expected: 3 ], ] ] ] 2025-09-01 Mon 02:02:25:774.564 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848854357504, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:02:25:775.630 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:25:775.680 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: SANDESH: Send FAILED: 1756692145775485 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.17 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:25:775.694 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:25:775.708 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:25:775.744 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: SANDESH: Send FAILED: 1756692145775586 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-3992-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.17 server_addrs= [ [ (*_iter6) = 10.20.0.17:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.17 server_addrs= [ [ (*_iter6) = 10.20.0.17:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.17, XMPP:dns-server:10.20.0.17, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:02:25:775.764 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:02:25:777.067 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848862754304, Pid 111888]: TCP [SYS_DEBUG]: TcpServerMessageLog: Server [fe80::f816:3eff:fe63:4c7b%vhost0]:8097 Initialization complete src/contrail-common/io/tcp_server.cc 137 2025-09-01 Mon 02:02:25:826.357 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:25:826.405 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: SANDESH: Send FAILED: 1756692145826278 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.17 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:25:826.414 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:25:826.621 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848858555904, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:25:826.644 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848858555904, Pid 111888]: SANDESH: Send FAILED: 1756692145826577 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.17 state = Connect event = Connect Event local_port = 45751 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:25:826.649 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848886536832, Pid 111888]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.17 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:02:25:826.652 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848858555904, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:25:826.923 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848858555904, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:25:826.941 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848858555904, Pid 111888]: SANDESH: Send FAILED: 1756692145826618 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.17 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:25:826.950 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848858555904, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:25:826.961 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848858555904, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:25:826.978 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848858555904, Pid 111888]: SANDESH: Send FAILED: 1756692145826731 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.17 state = Connect event = Connect Event local_port = 41401 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:25:826.984 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848858555904, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:25:827.841 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848886536832, Pid 111888]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.17 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:02:25:827.874 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848858555904, Pid 111888]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:02:25:827.955 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848862754304, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:25:827.987 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848858555904, Pid 111888]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:02:25:828.007 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848862754304, Pid 111888]: SANDESH: Send FAILED: 1756692145827928 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.17 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:25:828.025 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848862754304, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:25:828.040 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848862754304, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:25:828.061 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848862754304, Pid 111888]: SANDESH: Send FAILED: 1756692145828005 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.17 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:25:828.073 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848862754304, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:26:749.019 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: Processing scm::EvIdleHoldTimerExpired in state Idle 2025-09-01 Mon 02:02:26:749.306 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: Connect : Start Connect timer 10.0.0.254:8086 2025-09-01 Mon 02:02:26:749.344 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848886536832, Pid 111888]: -(-1) OnSessionEvent TCP Connect Failed 2025-09-01 Mon 02:02:26:749.380 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848886536832, Pid 111888]: -(-1):EvTcpConnectFail 2025-09-01 Mon 02:02:26:749.422 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: Processing scm::EvSandeshSend in state Connect 2025-09-01 Mon 02:02:26:749.453 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: SANDESH: Send FAILED: 1756692146749104 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-3992-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.17 server_addrs= [ [ (*_iter6) = 10.20.0.17:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.17 server_addrs= [ [ (*_iter6) = 10.20.0.17:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.254:8086, ] ] status = Initializing description = Idle : EvIdleHoldTimerExpired -> Connect ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.17, XMPP:dns-server:10.20.0.17, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:02:26:749.470 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: Wrong state: Connect for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:02:26:749.481 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: Processing scm::EvTcpConnectFail in state Connect 2025-09-01 Mon 02:02:26:749.488 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: Connect : EvTcpConnectFail 2025-09-01 Mon 02:02:26:749.544 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: TCP [SYS_ERR]: TcpSessionMessageLog: Session -(-1) > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:02:26:749.574 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: Idle 2025-09-01 Mon 02:02:26:749.626 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:26:749.761 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: SANDESH: Send FAILED: 1756692146749404 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3992-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Connect successful_connections = 0 pid = 111888 http_port = 8085 start_time = 1756692145743123 collector_name = collector_ip = 10.0.0.254:8086 collector_list= [ [ (*_iter40) = 10.0.0.254:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 5 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 4 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 8 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 8, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 19, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:02:26:749.786 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:02:26:749.797 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:26:749.814 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: SANDESH: Send FAILED: 1756692146749522 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-3992-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.17 server_addrs= [ [ (*_iter6) = 10.20.0.17:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.17 server_addrs= [ [ (*_iter6) = 10.20.0.17:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.17, XMPP:dns-server:10.20.0.17, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:02:26:749.824 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:02:26:749.832 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: Processing scm::EvTcpDeleteSession in state Idle 2025-09-01 Mon 02:02:26:749.866 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:26:749.967 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: SANDESH: Send FAILED: 1756692146749613 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3992-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Idle successful_connections = 0 pid = 111888 http_port = 8085 start_time = 1756692145743123 collector_name = collector_ip = 10.0.0.254:8086 collector_list= [ [ (*_iter40) = 10.0.0.254:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 6 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 4 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 4 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 8 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 9, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 20, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:02:26:749.983 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:02:26:755.122 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848879547904, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:26:755.173 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848879547904, Pid 111888]: SANDESH: Send FAILED: 1756692146755083 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.17 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:26:755.202 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848879547904, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:26:755.291 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848886536832, Pid 111888]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.17 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:02:26:755.300 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848866952704, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:26:755.344 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848866952704, Pid 111888]: SANDESH: Send FAILED: 1756692146755265 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.17 state = Connect event = Connect Event local_port = 46539 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:26:755.367 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848866952704, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:26:755.351 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848879547904, Pid 111888]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:02:26:755.428 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848854357504, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:26:755.458 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848854357504, Pid 111888]: SANDESH: Send FAILED: 1756692146755418 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.17 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:26:755.468 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848854357504, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:26:854.271 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848879547904, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:26:854.323 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848879547904, Pid 111888]: SANDESH: Send FAILED: 1756692146854238 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.17 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:26:854.334 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848879547904, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:26:854.493 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848886536832, Pid 111888]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.17 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:02:26:854.528 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848862754304, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:26:854.570 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848862754304, Pid 111888]: SANDESH: Send FAILED: 1756692146854475 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.17 state = Connect event = Connect Event local_port = 33759 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:26:854.579 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848862754304, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:26:854.698 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848879547904, Pid 111888]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:02:26:854.926 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848866952704, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:26:854.990 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848866952704, Pid 111888]: SANDESH: Send FAILED: 1756692146854816 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.17 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:26:855.022 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848866952704, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:28:727.164 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848879547904, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:28:727.276 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848879547904, Pid 111888]: SANDESH: Send FAILED: 1756692148727115 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.17 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:28:727.296 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848879547904, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:28:727.416 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848886536832, Pid 111888]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.17 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:02:28:727.426 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848854357504, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:28:727.515 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848854357504, Pid 111888]: SANDESH: Send FAILED: 1756692148727370 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.17 state = Connect event = Connect Event local_port = 35693 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:28:727.530 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848854357504, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:28:727.560 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848862754304, Pid 111888]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:02:28:727.679 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848854357504, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:28:727.703 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848854357504, Pid 111888]: SANDESH: Send FAILED: 1756692148727661 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.17 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:28:727.717 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848854357504, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:28:749.775 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848854357504, Pid 111888]: Processing scm::EvIdleHoldTimerExpired in state Idle 2025-09-01 Mon 02:02:28:750.111 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848854357504, Pid 111888]: Connect : Start Connect timer 10.0.0.254:8086 2025-09-01 Mon 02:02:28:750.140 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848886536832, Pid 111888]: -(-1) OnSessionEvent TCP Connect Failed 2025-09-01 Mon 02:02:28:750.169 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848886536832, Pid 111888]: -(-1):EvTcpConnectFail 2025-09-01 Mon 02:02:28:750.218 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848854357504, Pid 111888]: Processing scm::EvSandeshSend in state Connect 2025-09-01 Mon 02:02:28:750.246 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848854357504, Pid 111888]: SANDESH: Send FAILED: 1756692148749874 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-3992-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.17 server_addrs= [ [ (*_iter6) = 10.20.0.17:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.17 server_addrs= [ [ (*_iter6) = 10.20.0.17:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.254:8086, ] ] status = Initializing description = Idle : EvIdleHoldTimerExpired -> Connect ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.17, XMPP:dns-server:10.20.0.17, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:02:28:750.265 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848854357504, Pid 111888]: Wrong state: Connect for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:02:28:750.275 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848854357504, Pid 111888]: Processing scm::EvTcpConnectFail in state Connect 2025-09-01 Mon 02:02:28:750.282 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848854357504, Pid 111888]: Connect : EvTcpConnectFail 2025-09-01 Mon 02:02:28:750.324 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848854357504, Pid 111888]: TCP [SYS_ERR]: TcpSessionMessageLog: Session -(-1) > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:02:28:750.359 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848854357504, Pid 111888]: Idle 2025-09-01 Mon 02:02:28:750.412 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848854357504, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:28:750.554 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848854357504, Pid 111888]: SANDESH: Send FAILED: 1756692148750202 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3992-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Connect successful_connections = 0 pid = 111888 http_port = 8085 start_time = 1756692145743123 collector_name = collector_ip = 10.0.0.254:8086 collector_list= [ [ (*_iter40) = 10.0.0.254:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 7 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 6 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 7 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 17 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 5 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 15, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 32, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:02:28:750.572 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848854357504, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:02:28:750.582 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848854357504, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:28:750.599 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848854357504, Pid 111888]: SANDESH: Send FAILED: 1756692148750303 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-3992-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.17 server_addrs= [ [ (*_iter6) = 10.20.0.17:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.17 server_addrs= [ [ (*_iter6) = 10.20.0.17:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.17, XMPP:dns-server:10.20.0.17, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:02:28:750.609 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848854357504, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:02:28:750.618 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848854357504, Pid 111888]: Processing scm::EvTcpDeleteSession in state Idle 2025-09-01 Mon 02:02:28:750.649 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848854357504, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:28:750.750 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848854357504, Pid 111888]: SANDESH: Send FAILED: 1756692148750401 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3992-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Idle successful_connections = 0 pid = 111888 http_port = 8085 start_time = 1756692145743123 collector_name = collector_ip = 10.0.0.254:8086 collector_list= [ [ (*_iter40) = 10.0.0.254:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 8 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 6 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 8 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 17 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 5 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 16, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 33, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:02:28:750.769 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848854357504, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:02:28:807.774 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848858555904, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:28:807.857 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848858555904, Pid 111888]: SANDESH: Send FAILED: 1756692148807728 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.17 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:28:807.872 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848858555904, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:28:808.165 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848886536832, Pid 111888]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.17 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:02:28:808.179 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:28:808.382 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848879547904, Pid 111888]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:02:28:808.401 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: SANDESH: Send FAILED: 1756692148808109 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.17 state = Connect event = Connect Event local_port = 36605 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:28:808.497 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:28:808.636 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:28:808.658 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: SANDESH: Send FAILED: 1756692148808551 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.17 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:28:808.669 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:32:507.984 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:32:508.132 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: SANDESH: Send FAILED: 1756692152507919 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.17 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:32:508.159 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848886536832, Pid 111888]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.17 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:02:32:508.171 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:32:508.252 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:32:508.275 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: SANDESH: Send FAILED: 1756692152508214 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.17 state = Connect event = Connect Event local_port = 35123 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:32:508.284 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:32:508.306 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848854357504, Pid 111888]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:02:32:508.439 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:32:508.556 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: SANDESH: Send FAILED: 1756692152508332 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.17 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:32:508.614 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:32:508.648 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:32:508.680 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: SANDESH: Send FAILED: 1756692152508355 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.17 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:32:508.703 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:32:508.726 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:32:508.782 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: SANDESH: Send FAILED: 1756692152508531 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-3992-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.17 server_addrs= [ [ (*_iter6) = 10.20.0.17:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.17 server_addrs= [ [ (*_iter6) = 10.20.0.17:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.17, XMPP:dns-server:10.20.0.17, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:02:32:508.802 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:02:32:750.616 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848858555904, Pid 111888]: Processing scm::EvIdleHoldTimerExpired in state Idle 2025-09-01 Mon 02:02:32:751.048 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848858555904, Pid 111888]: Connect : Start Connect timer 10.0.0.254:8086 2025-09-01 Mon 02:02:32:751.089 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848886536832, Pid 111888]: -(-1) OnSessionEvent TCP Connect Failed 2025-09-01 Mon 02:02:32:751.154 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848886536832, Pid 111888]: -(-1):EvTcpConnectFail 2025-09-01 Mon 02:02:32:751.231 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848858555904, Pid 111888]: Processing scm::EvSandeshSend in state Connect 2025-09-01 Mon 02:02:32:751.315 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848858555904, Pid 111888]: SANDESH: Send FAILED: 1756692152750750 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-3992-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.17 server_addrs= [ [ (*_iter6) = 10.20.0.17:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.17 server_addrs= [ [ (*_iter6) = 10.20.0.17:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.254:8086, ] ] status = Initializing description = Idle : EvIdleHoldTimerExpired -> Connect ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.17, XMPP:dns-server:10.20.0.17, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:02:32:751.336 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848858555904, Pid 111888]: Wrong state: Connect for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:02:32:751.375 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848858555904, Pid 111888]: Processing scm::EvTcpConnectFail in state Connect 2025-09-01 Mon 02:02:32:751.393 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848858555904, Pid 111888]: Connect : EvTcpConnectFail 2025-09-01 Mon 02:02:32:751.485 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848858555904, Pid 111888]: TCP [SYS_ERR]: TcpSessionMessageLog: Session -(-1) > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:02:32:751.561 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848858555904, Pid 111888]: Idle 2025-09-01 Mon 02:02:32:751.678 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848858555904, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:32:752.054 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848858555904, Pid 111888]: SANDESH: Send FAILED: 1756692152751196 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3992-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Connect successful_connections = 0 pid = 111888 http_port = 8085 start_time = 1756692145743123 collector_name = collector_ip = 10.0.0.254:8086 collector_list= [ [ (*_iter40) = 10.0.0.254:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 10 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 8 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 10 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 24 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 7 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 20, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 44, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:02:32:752.099 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848858555904, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:02:32:752.124 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848858555904, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:32:752.215 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848858555904, Pid 111888]: SANDESH: Send FAILED: 1756692152751438 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-3992-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.17 server_addrs= [ [ (*_iter6) = 10.20.0.17:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.20.0.17 server_addrs= [ [ (*_iter6) = 10.20.0.17:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.17, XMPP:dns-server:10.20.0.17, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:02:32:752.239 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848858555904, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:02:32:752.259 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848858555904, Pid 111888]: Processing scm::EvTcpDeleteSession in state Idle 2025-09-01 Mon 02:02:32:752.324 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848858555904, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:32:752.637 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848858555904, Pid 111888]: SANDESH: Send FAILED: 1756692152751654 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3992-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Idle successful_connections = 0 pid = 111888 http_port = 8085 start_time = 1756692145743123 collector_name = collector_ip = 10.0.0.254:8086 collector_list= [ [ (*_iter40) = 10.0.0.254:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 11 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 8 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 11 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 24 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 7 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 21, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 45, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:02:32:752.686 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848858555904, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:02:33:092.983 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848858555904, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:33:093.066 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848858555904, Pid 111888]: SANDESH: Send FAILED: 1756692153092929 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.17 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:33:093.086 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848858555904, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:33:093.350 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848858555904, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:33:093.373 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848858555904, Pid 111888]: SANDESH: Send FAILED: 1756692153093299 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.17 state = Connect event = Connect Event local_port = 35597 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:33:093.386 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848858555904, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:33:093.395 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848886536832, Pid 111888]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.17 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:02:33:093.674 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:02:33:093.912 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848879547904, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:33:094.001 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848879547904, Pid 111888]: SANDESH: Send FAILED: 1756692153093777 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.17 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:33:094.028 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848879547904, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:33:094.054 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848879547904, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:33:094.085 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848879547904, Pid 111888]: SANDESH: Send FAILED: 1756692153093827 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.17 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:33:094.103 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848879547904, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:33:094.131 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848879547904, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:33:094.223 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848879547904, Pid 111888]: SANDESH: Send FAILED: 1756692153094000 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-3992-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.17 server_addrs= [ [ (*_iter6) = 10.20.0.17:5269, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.17 server_addrs= [ [ (*_iter6) = 10.20.0.17:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.17, XMPP:dns-server:10.20.0.17, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:02:33:094.244 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848879547904, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:02:40:212.534 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848866952704, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:40:212.608 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848866952704, Pid 111888]: SANDESH: Send FAILED: 1756692160212491 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.17 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:40:212.619 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848866952704, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:40:212.753 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848886536832, Pid 111888]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.17 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:02:40:212.760 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848854357504, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:40:212.824 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848854357504, Pid 111888]: SANDESH: Send FAILED: 1756692160212728 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.17 state = Connect event = Connect Event local_port = 40887 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:40:212.832 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848854357504, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:40:212.839 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:02:40:212.892 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848862754304, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:40:212.904 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848862754304, Pid 111888]: SANDESH: Send FAILED: 1756692160212882 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.17 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:40:212.911 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848862754304, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:40:212.917 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848862754304, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:40:212.925 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848862754304, Pid 111888]: SANDESH: Send FAILED: 1756692160212891 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.17 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:40:212.929 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848862754304, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:40:641.229 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848886536832, Pid 111888]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.0.0.254:8085::10.0.0.254:38312 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 255 2025-09-01 Mon 02:02:40:751.736 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848858555904, Pid 111888]: Processing scm::EvIdleHoldTimerExpired in state Idle 2025-09-01 Mon 02:02:40:752.302 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848858555904, Pid 111888]: Connect : Start Connect timer 10.0.0.254:8086 2025-09-01 Mon 02:02:40:752.325 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848886536832, Pid 111888]: -(-1) OnSessionEvent TCP Connect Failed 2025-09-01 Mon 02:02:40:752.388 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848886536832, Pid 111888]: -(-1):EvTcpConnectFail 2025-09-01 Mon 02:02:40:752.547 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848858555904, Pid 111888]: Processing scm::EvSandeshSend in state Connect 2025-09-01 Mon 02:02:40:752.622 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848858555904, Pid 111888]: SANDESH: Send FAILED: 1756692160751884 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-3992-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.17 server_addrs= [ [ (*_iter6) = 10.20.0.17:5269, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.17 server_addrs= [ [ (*_iter6) = 10.20.0.17:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.254:8086, ] ] status = Initializing description = Idle : EvIdleHoldTimerExpired -> Connect ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.17, XMPP:dns-server:10.20.0.17, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:02:40:752.663 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848858555904, Pid 111888]: Wrong state: Connect for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:02:40:752.690 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848858555904, Pid 111888]: Processing scm::EvTcpConnectFail in state Connect 2025-09-01 Mon 02:02:40:752.710 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848858555904, Pid 111888]: Connect : EvTcpConnectFail 2025-09-01 Mon 02:02:40:752.836 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848858555904, Pid 111888]: TCP [SYS_ERR]: TcpSessionMessageLog: Session -(-1) > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:02:40:752.903 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848858555904, Pid 111888]: Idle 2025-09-01 Mon 02:02:40:753.037 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848858555904, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:40:753.331 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848858555904, Pid 111888]: SANDESH: Send FAILED: 1756692160752487 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3992-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Connect successful_connections = 0 pid = 111888 http_port = 8085 start_time = 1756692145743123 collector_name = collector_ip = 10.0.0.254:8086 collector_list= [ [ (*_iter40) = 10.0.0.254:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 13 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 10 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 14 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 32 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 9 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 26, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 57, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:02:40:753.378 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848858555904, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:02:40:753.403 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848858555904, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:40:753.453 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848858555904, Pid 111888]: SANDESH: Send FAILED: 1756692160752765 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-3992-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.17 server_addrs= [ [ (*_iter6) = 10.20.0.17:5269, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.17 server_addrs= [ [ (*_iter6) = 10.20.0.17:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.17, XMPP:dns-server:10.20.0.17, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:02:40:753.479 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848858555904, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:02:40:753.526 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848858555904, Pid 111888]: Processing scm::EvTcpDeleteSession in state Idle 2025-09-01 Mon 02:02:40:753.594 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848858555904, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:40:753.896 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848858555904, Pid 111888]: SANDESH: Send FAILED: 1756692160753008 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3992-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Idle successful_connections = 0 pid = 111888 http_port = 8085 start_time = 1756692145743123 collector_name = collector_ip = 10.0.0.254:8086 collector_list= [ [ (*_iter40) = 10.0.0.254:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 14 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 10 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 15 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 32 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 9 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 27, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 58, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:02:40:753.939 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848858555904, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:02:41:086.187 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:41:086.364 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: SANDESH: Send FAILED: 1756692161086138 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.17 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:41:086.385 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:41:086.535 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848886536832, Pid 111888]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.17 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:02:41:086.572 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848879547904, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:41:086.718 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848858555904, Pid 111888]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:02:41:086.722 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848879547904, Pid 111888]: SANDESH: Send FAILED: 1756692161086497 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.17 state = Connect event = Connect Event local_port = 41047 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:41:086.817 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848879547904, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:41:086.940 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848862754304, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:41:087.026 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848862754304, Pid 111888]: SANDESH: Send FAILED: 1756692161086837 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.17 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:41:087.058 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848862754304, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:41:087.094 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848862754304, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:41:087.141 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848862754304, Pid 111888]: SANDESH: Send FAILED: 1756692161086909 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.17 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:41:087.172 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848862754304, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:53:525.574 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848886536832, Pid 111888]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.0.0.254:8085::10.0.0.254:33122 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 255 2025-09-01 Mon 02:02:54:613.313 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848879547904, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:54:613.414 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848879547904, Pid 111888]: SANDESH: Send FAILED: 1756692174613262 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.17 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:54:613.436 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848879547904, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:54:613.817 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848886536832, Pid 111888]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.17 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:02:54:613.876 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:54:613.932 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:02:54:614.036 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: SANDESH: Send FAILED: 1756692174613762 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.17 state = Connect event = Connect Event local_port = 35535 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:54:614.080 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:54:614.117 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:54:614.167 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: SANDESH: Send FAILED: 1756692174614054 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.17 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:54:614.193 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:54:614.223 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:54:614.263 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: SANDESH: Send FAILED: 1756692174614063 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.17 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:54:614.288 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:55:389.476 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848854357504, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:55:390.446 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848854357504, Pid 111888]: SANDESH: Send FAILED: 1756692175389403 [SYS_NOTICE]: UveVirtualNetworkAgentTrace: data= [ name = __UNKNOWN__ vrf_stats_list= [ [ [ name = __untitled__ diag_packet_count = 0 unknown_unicast_floods = 0 arp_packet_counts= [ from_vm_interface= [ stats= [ proxies = 0 stitches = 0 floods = 0 ] ] from_physical_interface= [ stats= [ proxies = 0 stitches = 0 floods = 0 ] ] ] offload_packet_counts= [ gro = 0 ] nh_packet_counts= [ discards = 0 resolves = 0 l3_receives = 0 l2_receives = 0 local_vm_l3_forwards = 0 local_vm_l2_forwards = 0 vrf_translates = 0 ecmp_forwards = 0 tunnel_nh_stats= [ udp_encaps = 0 mpls_over_udp_encaps = 0 mpls_over_gre_encaps = 0 vxlan_encaps = 0 ] comp_nh_stats= [ source_replication_forwards = 0 edge_replication_forwards = 0 local_vm_l3_forwards = 0 total_multicast_forwards = 0 ] ] ], ] ] ] 2025-09-01 Mon 02:02:55:390.478 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848854357504, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: UveVirtualNetworkAgentTrace 2025-09-01 Mon 02:02:55:390.972 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848858555904, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:55:391.862 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848858555904, Pid 111888]: SANDESH: Send FAILED: 1756692175390882 [SYS_NOTICE]: UveVrouterAgent: data= [ name = cn-jenkins-deploy-platform-ansible-os-3992-1 interface_list= [ [ ] ] error_intf_list= [ [ ] ] no_config_intf_list= [ [ (*_iter151) = 00000000-0000-0000-0000-000000000000, ] ] vmi_count= [ active = 1 ] down_interface_count = 1 ] 2025-09-01 Mon 02:02:55:391.905 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848858555904, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: UveVrouterAgent 2025-09-01 Mon 02:02:55:746.726 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848866952704, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:55:746.883 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848866952704, Pid 111888]: SANDESH: Send FAILED: 1756692175746454 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3992-1:Compute:contrail-vrouter-agent:0 sm_queue_count = 0 sm_stats= [ ev_stats= [ [ [ event = scm::EvCollectorUpdate enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvIdleHoldTimerExpired enqueues = 4 dequeues = 4 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvSandeshSend enqueues = 71 dequeues = 71 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvStart enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpConnectFail enqueues = 5 dequeues = 5 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpDeleteSession enqueues = 5 dequeues = 5 enqueue_fails = 0 dequeue_fails = 0 ], [ event = enqueues = 87 dequeues = 87 enqueue_fails = 0 dequeue_fails = 0 ], ] ] state = Idle state_since = 1756692160752900 last_state = Idle last_event = scm::EvSandeshSend last_event_at = 1756692175390957 ] max_sm_queue_count = 6 ] 2025-09-01 Mon 02:02:55:746.912 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848866952704, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:02:55:746.931 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848866952704, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:55:747.125 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848866952704, Pid 111888]: SANDESH: Send FAILED: 1756692175746660 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3992-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Idle successful_connections = 0 pid = 111888 http_port = 8085 start_time = 1756692145743123 collector_name = collector_ip = 10.0.0.254:8086 collector_list= [ [ (*_iter40) = 10.0.0.254:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 15 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 12 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 18 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 40 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 11 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 32, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 71, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:02:55:747.168 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848866952704, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:02:55:750.446 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848879547904, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:55:750.556 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848879547904, Pid 111888]: SANDESH: Send FAILED: 1756692175750367 [SYS_NOTICE]: UveVrouterAgent: data= [ name = cn-jenkins-deploy-platform-ansible-os-3992-1 xmpp_peer_list= [ [ [ ip = 10.20.0.17 status = 0 setup_time = 0 primary = 1 ], ] ] self_ip_list= [ [ (*_iter146) = 10.20.0.17, ] ] build_info = {"build-info":[{"build-version":"v0+nightly","build-time":"2025-09-01 00:13:09.128962","build-user":"root","build-hostname":"jenkins-build-prepare-sandbox-10717-1.novalocal","build-id":"unknown","build-number":"unknown"}]} phy_if= [ [ [ name = ens6 mac_address = fa:16:3e:63:4c:7b ], ] ] vhost_if= [ name = vhost0 mac_address = fa:16:3e:63:4c:7b ] log_file = /var/log/contrail/vrouter-agent/contrail-vrouter-agent.log config_file = /etc/contrail/contrail-vrouter-agent.conf log_local = 1 log_level = SYS_DEBUG log_category = sandesh_http_port = 8085 dns_servers= [ [ (*_iter149) = 10.20.0.17, ] ] vhost_cfg= [ name = vhost0 ip = 10.20.0.17 ip_prefix_len = 24 gateway = 0.0.0.0 ] eth_name= [ [ (*_iter150) = ens6, ] ] tunnel_type = MPLSoGRE hypervisor = kvm control_ip = 10.20.0.17 dns_server_list_cfg= [ [ (*_iter152) = 10.20.0.17:53, ] ] control_node_list_cfg= [ [ (*_iter153) = 10.20.0.17:5269, ] ] hostname_cfg = cn-jenkins-deploy-platform-ansible-os-3992-1 flow_cache_timeout_cfg = 0 ll_max_system_flows_cfg = 16384 ll_max_vm_flows_cfg = 16384 gateway_cfg_list= [ [ ] ] headless_mode_cfg = 1 collector_server_list_cfg= [ [ (*_iter155) = 10.0.0.254:8086, ] ] log_flow = 0 mode = VROUTER platform = HOST vr_limits= [ max_labels = 5120 max_nexthops = 524288 max_interfaces = 4355 max_vrfs = 4096 max_mirror_entries = 255 vrouter_max_bridge_entries = 262144 vrouter_max_oflow_bridge_entries = 53248 vrouter_build_info = {"build-info": [{"build-version": "v0+nightly", "build-time": "2025-09-01 00:11:15.602301", "build-user": "root", "build-hostname": "jenkins-build-prepare-sandbox-10717-1.novalocal"}]} vrouter_max_flow_entries = 524288 vrouter_max_oflow_entries = 105472 vrouter_priority_tagging = 1 ] bgpaas_enabled = 0 port_mirror_enabled = 0 subcluster_name = vr_high_watermark = 80 vr_low_watermark = 75 vr_limit_exceeded_list= [ [ ] ] res_limit = 0 res_table_limit = 0 loopback_ip = 0.0.0.0 gateway_list= [ [ (*_iter161) = 0.0.0.0, ] ] ] 2025-09-01 Mon 02:02:55:750.586 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848879547904, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: UveVrouterAgent 2025-09-01 Mon 02:02:55:750.597 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848879547904, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:55:750.696 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848879547904, Pid 111888]: SANDESH: Send FAILED: 1756692175750446 [SYS_NOTICE]: VrouterStats: data= [ name = cn-jenkins-deploy-platform-ansible-os-3992-1 in_tpkts = 20 in_bytes = 924 out_tpkts = 18 out_bytes = 784 exception_packets = 8 exception_packets_dropped = 8 total_flows = 0 aged_flows = 0 uptime = 1756692145385467 phy_if_info= [ [ _iter254->first = ens6 [ speed = -1 duplexity = 255 ], ] ] raw_vhost_stats= [ name = vhost0 in_pkts = 20 in_bytes = 924 out_pkts = 18 out_bytes = 784 speed = -1 duplexity = -1 ] raw_vhost_drop_stats= [ ds_discard = 0 ds_pull = 0 ds_invalid_if = 0 ds_invalid_arp = 0 ds_trap_no_if = 0 ds_nowhere_to_go = 0 ds_flow_queue_limit_exceeded = 0 ds_flow_no_memory = 0 ds_flow_invalid_protocol = 0 ds_flow_nat_no_rflow = 0 ds_flow_action_drop = 0 ds_flow_action_invalid = 0 ds_flow_unusable = 0 ds_flow_table_full = 0 ds_interface_tx_discard = 0 ds_interface_drop = 0 ds_duplicated = 0 ds_push = 0 ds_ttl_exceeded = 0 ds_invalid_nh = 0 ds_invalid_label = 0 ds_invalid_protocol = 0 ds_interface_rx_discard = 0 ds_invalid_mcast_source = 0 ds_head_alloc_fail = 0 ds_pcow_fail = 0 ds_mcast_clone_fail = 0 ds_rewrite_fail = 0 ds_misc = 0 ds_invalid_packet = 0 ds_cksum_err = 0 ds_no_fmd = 0 ds_invalid_vnid = 0 ds_frag_err = 0 ds_invalid_source = 0 ds_mcast_df_bit = 0 ds_l2_no_route = 0 ds_vlan_fwd_tx = 0 ds_vlan_fwd_enq = 0 ds_drop_new_flow = 0 ds_flow_evict = 0 ds_trap_original = 0 ds_no_memory = 0 ds_fragment_queue_fail = 0 ds_no_frag_entry = 0 ds_icmp_error = 0 ds_clone_fail = 0 ds_drop_pkts = 0 ] raw_drop_stats= [ ds_discard = 0 ds_pull = 0 ds_invalid_if = 0 ds_invalid_arp = 0 ds_trap_no_if = 0 ds_nowhere_to_go = 0 ds_flow_queue_limit_exceeded = 0 ds_flow_no_memory = 0 ds_flow_invalid_protocol = 0 ds_flow_nat_no_rflow = 0 ds_flow_action_drop = 0 ds_flow_action_invalid = 0 ds_flow_unusable = 0 ds_flow_table_full = 0 ds_interface_tx_discard = 0 ds_interface_drop = 0 ds_duplicated = 0 ds_push = 0 ds_ttl_exceeded = 0 ds_invalid_nh = 0 ds_invalid_label = 0 ds_invalid_protocol = 0 ds_interface_rx_discard = 0 ds_invalid_mcast_source = 0 ds_head_alloc_fail = 0 ds_pcow_fail = 0 ds_mcast_clone_fail = 0 ds_rewrite_fail = 0 ds_misc = 0 ds_invalid_packet = 0 ds_cksum_err = 0 ds_no_fmd = 0 ds_invalid_vnid = 0 ds_frag_err = 0 ds_invalid_source = 0 ds_mcast_df_bit = 0 ds_l2_no_route = 0 ds_vlan_fwd_tx = 0 ds_vlan_fwd_enq = 0 ds_drop_new_flow = 0 ds_flow_evict = 0 ds_trap_original = 0 ds_no_memory = 0 ds_fragment_queue_fail = 0 ds_no_frag_entry = 0 ds_icmp_error = 0 ds_clone_fail = 0 ds_drop_pkts = 0 ] flow_rate= [ added_flows = 0 max_flow_adds_per_second = 0 min_flow_adds_per_second = 0 deleted_flows = 0 max_flow_deletes_per_second = 0 min_flow_deletes_per_second = 0 active_flows = 0 hold_flows = 0 ] raw_phy_if_stats= [ [ _iter262->first = ens6 [ in_pkts = 1 in_bytes = 70 out_pkts = 20 out_bytes = 924 ], ] ] raw_phy_if_drop_stats= [ [ _iter263->first = ens6 [ ds_discard = 0 ds_pull = 0 ds_invalid_if = 0 ds_invalid_arp = 0 ds_trap_no_if = 0 ds_nowhere_to_go = 0 ds_flow_queue_limit_exceeded = 0 ds_flow_no_memory = 0 ds_flow_invalid_protocol = 0 ds_flow_nat_no_rflow = 0 ds_flow_action_drop = 0 ds_flow_action_invalid = 0 ds_flow_unusable = 0 ds_flow_table_full = 0 ds_interface_tx_discard = 0 ds_interface_drop = 0 ds_duplicated = 0 ds_push = 0 ds_ttl_exceeded = 0 ds_invalid_nh = 0 ds_invalid_label = 0 ds_invalid_protocol = 0 ds_interface_rx_discard = 0 ds_invalid_mcast_source = 0 ds_head_alloc_fail = 0 ds_pcow_fail = 0 ds_mcast_clone_fail = 0 ds_rewrite_fail = 0 ds_misc = 0 ds_invalid_packet = 0 ds_cksum_err = 0 ds_no_fmd = 0 ds_invalid_vnid = 0 ds_frag_err = 0 ds_invalid_source = 0 ds_mcast_df_bit = 0 ds_l2_no_route = 0 ds_vlan_fwd_tx = 0 ds_vlan_fwd_enq = 0 ds_drop_new_flow = 0 ds_flow_evict = 0 ds_trap_original = 0 ds_no_memory = 0 ds_fragment_queue_fail = 0 ds_no_frag_entry = 0 ds_icmp_error = 0 ds_clone_fail = 0 ds_drop_pkts = 0 ], ] ] ] 2025-09-01 Mon 02:02:55:750.713 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848879547904, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: VrouterStats 2025-09-01 Mon 02:02:55:750.721 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848879547904, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:55:750.745 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848879547904, Pid 111888]: SANDESH: Send FAILED: 1756692175750532 [SYS_NOTICE]: VrouterControlStatsTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3992-1 raw_xmpp_stats= [ [ _iter306->first = 10.20.0.17 [ reconnects = 0 in_msgs = 0 out_msgs = 0 ], ] ] raw_ifmap_stats= [ [ _iter307->first = link_delete_parse_errors _iter307->second = 0, _iter307->first = link_update_parse_errors _iter307->second = 0, _iter307->first = node_delete_parse_errors _iter307->second = 0, _iter307->first = node_update_parse_errors _iter307->second = 0, ] ] raw_rt_table_size= [ [ _iter308->first = default-domain:default-project:ip-fabric:__default__ [ inet4_unicast = 4 inet4_multicast = 0 evpn = 0 bridge = 2 inet6_unicast = 2 ], _iter308->first = default-domain:default-project:ip-fabric:ip-fabric [ inet4_unicast = 2 inet4_multicast = 0 evpn = 1 bridge = 2 inet6_unicast = 0 ], ] ] ] 2025-09-01 Mon 02:02:55:750.752 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848879547904, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: VrouterControlStatsTrace 2025-09-01 Mon 02:02:56:753.053 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848879547904, Pid 111888]: Processing scm::EvIdleHoldTimerExpired in state Idle 2025-09-01 Mon 02:02:56:753.382 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848879547904, Pid 111888]: Connect : Start Connect timer 10.0.0.254:8086 2025-09-01 Mon 02:02:56:753.417 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848886536832, Pid 111888]: -(-1) OnSessionEvent TCP Connect Failed 2025-09-01 Mon 02:02:56:753.459 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848886536832, Pid 111888]: -(-1):EvTcpConnectFail 2025-09-01 Mon 02:02:56:753.598 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848879547904, Pid 111888]: Processing scm::EvSandeshSend in state Connect 2025-09-01 Mon 02:02:56:753.645 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848879547904, Pid 111888]: SANDESH: Send FAILED: 1756692176753135 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-3992-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.17 server_addrs= [ [ (*_iter6) = 10.20.0.17:5269, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.17 server_addrs= [ [ (*_iter6) = 10.20.0.17:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.254:8086, ] ] status = Initializing description = Idle : EvIdleHoldTimerExpired -> Connect ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.17, XMPP:dns-server:10.20.0.17, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:02:56:753.703 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848879547904, Pid 111888]: Wrong state: Connect for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:02:56:753.719 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848879547904, Pid 111888]: Processing scm::EvTcpConnectFail in state Connect 2025-09-01 Mon 02:02:56:753.731 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848879547904, Pid 111888]: Connect : EvTcpConnectFail 2025-09-01 Mon 02:02:56:753.797 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848879547904, Pid 111888]: TCP [SYS_ERR]: TcpSessionMessageLog: Session -(-1) > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:02:56:753.837 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848879547904, Pid 111888]: Idle 2025-09-01 Mon 02:02:56:753.926 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848879547904, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:56:754.099 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848879547904, Pid 111888]: SANDESH: Send FAILED: 1756692176753555 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3992-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Connect successful_connections = 0 pid = 111888 http_port = 8085 start_time = 1756692145743123 collector_name = collector_ip = 10.0.0.254:8086 collector_list= [ [ (*_iter40) = 10.0.0.254:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 15 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 14 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 18 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterControlStatsTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 40 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 11 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 32, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 76, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:02:56:754.181 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848879547904, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:02:56:754.195 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848879547904, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:56:754.271 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848879547904, Pid 111888]: SANDESH: Send FAILED: 1756692176753764 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-3992-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.17 server_addrs= [ [ (*_iter6) = 10.20.0.17:5269, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.17 server_addrs= [ [ (*_iter6) = 10.20.0.17:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.17, XMPP:dns-server:10.20.0.17, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:02:56:754.309 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848879547904, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:02:56:754.318 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848879547904, Pid 111888]: Processing scm::EvTcpDeleteSession in state Idle 2025-09-01 Mon 02:02:56:754.374 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848879547904, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:56:754.564 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848879547904, Pid 111888]: SANDESH: Send FAILED: 1756692176753906 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3992-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Idle successful_connections = 0 pid = 111888 http_port = 8085 start_time = 1756692145743123 collector_name = collector_ip = 10.0.0.254:8086 collector_list= [ [ (*_iter40) = 10.0.0.254:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 16 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 14 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 19 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterControlStatsTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 40 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 11 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 33, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 77, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:02:56:754.591 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848879547904, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:02:57:935.288 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:57:935.414 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: SANDESH: Send FAILED: 1756692177935215 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.17 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:57:935.442 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:57:935.644 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848854357504, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:57:935.667 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848886536832, Pid 111888]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.17 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:02:57:935.795 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848854357504, Pid 111888]: SANDESH: Send FAILED: 1756692177935576 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.17 state = Connect event = Connect Event local_port = 34087 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:57:935.938 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848854357504, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:57:936.035 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848854357504, Pid 111888]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:02:57:936.151 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848858555904, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:57:936.203 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848858555904, Pid 111888]: SANDESH: Send FAILED: 1756692177936114 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.17 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:57:936.218 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848858555904, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:02:57:936.231 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848858555904, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:02:57:936.247 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848858555904, Pid 111888]: SANDESH: Send FAILED: 1756692177936143 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.17 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:02:57:936.257 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848858555904, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:06:541.937 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848886536832, Pid 111888]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.0.0.254:8085::10.0.0.254:34404 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 255 2025-09-01 Mon 02:03:19:536.339 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848886536832, Pid 111888]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.0.0.254:8085::10.0.0.254:36776 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 255 2025-09-01 Mon 02:03:23:234.437 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:23:234.599 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: SANDESH: Send FAILED: 1756692203234385 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.17 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:23:234.636 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:23:234.664 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:23:234.700 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: SANDESH: Send FAILED: 1756692203234649 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.17 state = Connect event = Connect Event local_port = 46355 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:23:234.719 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:23:234.760 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848886536832, Pid 111888]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.17 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:03:23:234.973 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848854357504, Pid 111888]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:03:23:235.099 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848879547904, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:23:235.190 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848879547904, Pid 111888]: SANDESH: Send FAILED: 1756692203235053 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.17 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:23:235.221 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848879547904, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:23:235.247 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848879547904, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:23:235.277 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848879547904, Pid 111888]: SANDESH: Send FAILED: 1756692203235064 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.17 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:23:235.295 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848879547904, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:24:544.894 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848886536832, Pid 111888]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.20.0.17:8085::10.20.0.17:33440 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 255 2025-09-01 Mon 02:03:24:564.605 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848886536832, Pid 111888]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.20.0.17:8085::10.20.0.17:33456 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 255 2025-09-01 Mon 02:03:25:747.260 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:25:747.523 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: SANDESH: Send FAILED: 1756692205746958 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3992-1:Compute:contrail-vrouter-agent:0 sm_queue_count = 0 sm_stats= [ ev_stats= [ [ [ event = scm::EvCollectorUpdate enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvIdleHoldTimerExpired enqueues = 5 dequeues = 5 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvSandeshSend enqueues = 88 dequeues = 88 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvStart enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpConnectFail enqueues = 6 dequeues = 6 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpDeleteSession enqueues = 6 dequeues = 6 enqueue_fails = 0 dequeue_fails = 0 ], [ event = enqueues = 107 dequeues = 107 enqueue_fails = 0 dequeue_fails = 0 ], ] ] state = Idle state_since = 1756692176753836 last_state = Idle last_event = scm::EvSandeshSend last_event_at = 1756692203235243 ] max_sm_queue_count = 6 ] 2025-09-01 Mon 02:03:25:747.582 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:03:25:747.618 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:25:748.028 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: SANDESH: Send FAILED: 1756692205747169 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3992-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Idle successful_connections = 0 pid = 111888 http_port = 8085 start_time = 1756692145743123 collector_name = collector_ip = 10.0.0.254:8086 collector_list= [ [ (*_iter40) = 10.0.0.254:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 17 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 16 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 25 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterControlStatsTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 48 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 13 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 41, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 88, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:03:25:748.099 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:03:25:752.940 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:25:753.094 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: SANDESH: Send FAILED: 1756692205752790 [SYS_NOTICE]: VrouterStats: data= [ name = cn-jenkins-deploy-platform-ansible-os-3992-1 cpu_info= [ num_cpu = 8 sys_mem_info= [ total = 32 used = 0 free = 0 buffers = 0 cached = 510398136 node_type = ] meminfo= [ virt = 32561 peakvirt = 0 res = 0 ] cpuload= [ one_min_avg = 0.15125 five_min_avg = 0.27375 fifteen_min_avg = 0.24 ] cpu_share = 0.494597 ] ] 2025-09-01 Mon 02:03:25:753.125 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: VrouterStats 2025-09-01 Mon 02:03:25:753.155 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:25:753.234 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: SANDESH: Send FAILED: 1756692205752872 [SYS_NOTICE]: ComputeCpuStateTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3992-1 cpu_info= [ [ [ mem_virt = 32561 cpu_share = 0.494597 used_sys_mem = 3784569160 one_min_cpuload = 0.15125 mem_res = 0 ], ] ] ] 2025-09-01 Mon 02:03:25:753.267 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: ComputeCpuStateTrace 2025-09-01 Mon 02:03:25:753.290 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:25:753.638 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: SANDESH: Send FAILED: 1756692205752922 [SYS_NOTICE]: VrouterStats: data= [ name = cn-jenkins-deploy-platform-ansible-os-3992-1 in_tpkts = 35 in_bytes = 1554 out_tpkts = 37 out_bytes = 1694 exception_packets = 9 exception_packets_dropped = 9 raw_vhost_stats= [ name = vhost0 in_pkts = 35 in_bytes = 1554 out_pkts = 37 out_bytes = 1694 speed = -1 duplexity = -1 ] raw_vhost_drop_stats= [ ds_discard = 0 ds_pull = 0 ds_invalid_if = 0 ds_invalid_arp = 0 ds_trap_no_if = 0 ds_nowhere_to_go = 0 ds_flow_queue_limit_exceeded = 0 ds_flow_no_memory = 0 ds_flow_invalid_protocol = 0 ds_flow_nat_no_rflow = 0 ds_flow_action_drop = 0 ds_flow_action_invalid = 0 ds_flow_unusable = 0 ds_flow_table_full = 0 ds_interface_tx_discard = 0 ds_interface_drop = 0 ds_duplicated = 0 ds_push = 0 ds_ttl_exceeded = 0 ds_invalid_nh = 0 ds_invalid_label = 0 ds_invalid_protocol = 0 ds_interface_rx_discard = 0 ds_invalid_mcast_source = 0 ds_head_alloc_fail = 0 ds_pcow_fail = 0 ds_mcast_clone_fail = 0 ds_rewrite_fail = 0 ds_misc = 0 ds_invalid_packet = 0 ds_cksum_err = 0 ds_no_fmd = 0 ds_invalid_vnid = 0 ds_frag_err = 0 ds_invalid_source = 0 ds_mcast_df_bit = 0 ds_l2_no_route = 0 ds_vlan_fwd_tx = 0 ds_vlan_fwd_enq = 0 ds_drop_new_flow = 0 ds_flow_evict = 0 ds_trap_original = 0 ds_no_memory = 0 ds_fragment_queue_fail = 0 ds_no_frag_entry = 0 ds_icmp_error = 0 ds_clone_fail = 0 ds_drop_pkts = 0 ] raw_drop_stats= [ ds_discard = 0 ds_pull = 0 ds_invalid_if = 0 ds_invalid_arp = 0 ds_trap_no_if = 0 ds_nowhere_to_go = 0 ds_flow_queue_limit_exceeded = 0 ds_flow_no_memory = 0 ds_flow_invalid_protocol = 0 ds_flow_nat_no_rflow = 0 ds_flow_action_drop = 0 ds_flow_action_invalid = 0 ds_flow_unusable = 0 ds_flow_table_full = 0 ds_interface_tx_discard = 0 ds_interface_drop = 0 ds_duplicated = 0 ds_push = 0 ds_ttl_exceeded = 0 ds_invalid_nh = 0 ds_invalid_label = 0 ds_invalid_protocol = 0 ds_interface_rx_discard = 0 ds_invalid_mcast_source = 0 ds_head_alloc_fail = 0 ds_pcow_fail = 0 ds_mcast_clone_fail = 0 ds_rewrite_fail = 0 ds_misc = 0 ds_invalid_packet = 0 ds_cksum_err = 0 ds_no_fmd = 0 ds_invalid_vnid = 0 ds_frag_err = 0 ds_invalid_source = 0 ds_mcast_df_bit = 0 ds_l2_no_route = 0 ds_vlan_fwd_tx = 0 ds_vlan_fwd_enq = 0 ds_drop_new_flow = 0 ds_flow_evict = 0 ds_trap_original = 0 ds_no_memory = 0 ds_fragment_queue_fail = 0 ds_no_frag_entry = 0 ds_icmp_error = 0 ds_clone_fail = 0 ds_drop_pkts = 0 ] flow_rate= [ added_flows = 0 max_flow_adds_per_second = 0 min_flow_adds_per_second = 0 deleted_flows = 0 max_flow_deletes_per_second = 0 min_flow_deletes_per_second = 0 active_flows = 0 hold_flows = 0 ] raw_phy_if_stats= [ [ _iter262->first = ens6 [ in_pkts = 5 in_bytes = 350 out_pkts = 35 out_bytes = 1554 ], ] ] raw_phy_if_drop_stats= [ [ _iter263->first = ens6 [ ds_discard = 0 ds_pull = 0 ds_invalid_if = 0 ds_invalid_arp = 0 ds_trap_no_if = 0 ds_nowhere_to_go = 0 ds_flow_queue_limit_exceeded = 0 ds_flow_no_memory = 0 ds_flow_invalid_protocol = 0 ds_flow_nat_no_rflow = 0 ds_flow_action_drop = 0 ds_flow_action_invalid = 0 ds_flow_unusable = 0 ds_flow_table_full = 0 ds_interface_tx_discard = 0 ds_interface_drop = 0 ds_duplicated = 0 ds_push = 0 ds_ttl_exceeded = 0 ds_invalid_nh = 0 ds_invalid_label = 0 ds_invalid_protocol = 0 ds_interface_rx_discard = 0 ds_invalid_mcast_source = 0 ds_head_alloc_fail = 0 ds_pcow_fail = 0 ds_mcast_clone_fail = 0 ds_rewrite_fail = 0 ds_misc = 0 ds_invalid_packet = 0 ds_cksum_err = 0 ds_no_fmd = 0 ds_invalid_vnid = 0 ds_frag_err = 0 ds_invalid_source = 0 ds_mcast_df_bit = 0 ds_l2_no_route = 0 ds_vlan_fwd_tx = 0 ds_vlan_fwd_enq = 0 ds_drop_new_flow = 0 ds_flow_evict = 0 ds_trap_original = 0 ds_no_memory = 0 ds_fragment_queue_fail = 0 ds_no_frag_entry = 0 ds_icmp_error = 0 ds_clone_fail = 0 ds_drop_pkts = 0 ], ] ] ] 2025-09-01 Mon 02:03:25:753.765 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: VrouterStats 2025-09-01 Mon 02:03:25:753.792 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:25:753.865 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: SANDESH: Send FAILED: 1756692205752975 [SYS_NOTICE]: VrouterControlStatsTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3992-1 raw_xmpp_stats= [ [ _iter306->first = 10.20.0.17 [ reconnects = 0 in_msgs = 0 out_msgs = 0 ], ] ] raw_ifmap_stats= [ [ _iter307->first = link_delete_parse_errors _iter307->second = 0, _iter307->first = link_update_parse_errors _iter307->second = 0, _iter307->first = node_delete_parse_errors _iter307->second = 0, _iter307->first = node_update_parse_errors _iter307->second = 0, ] ] raw_rt_table_size= [ [ _iter308->first = default-domain:default-project:ip-fabric:__default__ [ inet4_unicast = 4 inet4_multicast = 0 evpn = 0 bridge = 2 inet6_unicast = 2 ], _iter308->first = default-domain:default-project:ip-fabric:ip-fabric [ inet4_unicast = 2 inet4_multicast = 0 evpn = 1 bridge = 2 inet6_unicast = 0 ], ] ] ] 2025-09-01 Mon 02:03:25:753.889 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: VrouterControlStatsTrace 2025-09-01 Mon 02:03:26:754.058 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: Processing scm::EvIdleHoldTimerExpired in state Idle 2025-09-01 Mon 02:03:26:754.654 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: Connect : Start Connect timer 10.0.0.254:8086 2025-09-01 Mon 02:03:26:754.670 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848886536832, Pid 111888]: -(-1) OnSessionEvent TCP Connect Failed 2025-09-01 Mon 02:03:26:754.754 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848886536832, Pid 111888]: -(-1):EvTcpConnectFail 2025-09-01 Mon 02:03:26:754.981 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: Processing scm::EvSandeshSend in state Connect 2025-09-01 Mon 02:03:26:755.083 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: SANDESH: Send FAILED: 1756692206754212 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-3992-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.17 server_addrs= [ [ (*_iter6) = 10.20.0.17:5269, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.17 server_addrs= [ [ (*_iter6) = 10.20.0.17:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.254:8086, ] ] status = Initializing description = Idle : EvIdleHoldTimerExpired -> Connect ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.17, XMPP:dns-server:10.20.0.17, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:03:26:755.142 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: Wrong state: Connect for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:03:26:755.172 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: Processing scm::EvTcpConnectFail in state Connect 2025-09-01 Mon 02:03:26:755.193 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: Connect : EvTcpConnectFail 2025-09-01 Mon 02:03:26:755.306 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: TCP [SYS_ERR]: TcpSessionMessageLog: Session -(-1) > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:03:26:755.382 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: Idle 2025-09-01 Mon 02:03:26:755.574 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:26:756.179 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: SANDESH: Send FAILED: 1756692206754894 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3992-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Connect successful_connections = 0 pid = 111888 http_port = 8085 start_time = 1756692145743123 collector_name = collector_ip = 10.0.0.254:8086 collector_list= [ [ (*_iter40) = 10.0.0.254:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ComputeCpuStateTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 17 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 18 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 25 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterControlStatsTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 3 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 48 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 13 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 41, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 94, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:03:26:756.259 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:03:26:756.305 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:26:756.383 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: SANDESH: Send FAILED: 1756692206755253 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-3992-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.17 server_addrs= [ [ (*_iter6) = 10.20.0.17:5269, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.17 server_addrs= [ [ (*_iter6) = 10.20.0.17:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.17, XMPP:dns-server:10.20.0.17, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:03:26:756.411 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:03:26:756.469 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: Processing scm::EvTcpDeleteSession in state Idle 2025-09-01 Mon 02:03:26:756.565 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:26:756.949 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: SANDESH: Send FAILED: 1756692206755497 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3992-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Idle successful_connections = 0 pid = 111888 http_port = 8085 start_time = 1756692145743123 collector_name = collector_ip = 10.0.0.254:8086 collector_list= [ [ (*_iter40) = 10.0.0.254:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ComputeCpuStateTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 18 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 18 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 26 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterControlStatsTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 3 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 48 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 13 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 42, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 95, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:03:26:756.996 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:03:29:256.478 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848854357504, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:29:256.556 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848854357504, Pid 111888]: SANDESH: Send FAILED: 1756692209256428 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.17 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:29:256.569 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848854357504, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:29:256.694 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848858555904, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:29:256.714 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848858555904, Pid 111888]: SANDESH: Send FAILED: 1756692209256684 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.17 state = Connect event = Connect Event local_port = 40811 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:29:256.726 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848858555904, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:29:256.750 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848886536832, Pid 111888]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.17 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:03:29:256.847 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848862754304, Pid 111888]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:03:29:256.900 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848854357504, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:29:256.914 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848854357504, Pid 111888]: SANDESH: Send FAILED: 1756692209256893 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.17 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:29:256.922 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848854357504, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:29:256.945 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848854357504, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:29:256.958 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848854357504, Pid 111888]: SANDESH: Send FAILED: 1756692209256898 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.17 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:29:256.966 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848854357504, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:32:399.805 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848886536832, Pid 111888]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.0.0.254:8085::10.0.0.254:35546 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 255 2025-09-01 Mon 02:03:45:737.040 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848886536832, Pid 111888]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.0.0.254:8085::10.0.0.254:45504 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 255 2025-09-01 Mon 02:03:51:315.586 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848858555904, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:51:315.846 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848858555904, Pid 111888]: SANDESH: Send FAILED: 1756692231315448 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.17 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:51:315.954 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848858555904, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:51:316.100 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848886536832, Pid 111888]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.17 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:03:51:316.123 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848862754304, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:51:316.303 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848866952704, Pid 111888]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:03:51:316.332 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848862754304, Pid 111888]: SANDESH: Send FAILED: 1756692231316024 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.17 state = Connect event = Connect Event local_port = 43379 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:51:316.374 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848862754304, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:51:316.414 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848862754304, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:51:316.461 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848862754304, Pid 111888]: SANDESH: Send FAILED: 1756692231316383 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.17 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:51:316.490 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848862754304, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:51:316.553 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848862754304, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:51:316.630 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848862754304, Pid 111888]: SANDESH: Send FAILED: 1756692231316398 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.17 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:03:51:316.661 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848862754304, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:03:55:747.927 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848866952704, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:55:748.119 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848866952704, Pid 111888]: SANDESH: Send FAILED: 1756692235747483 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3992-1:Compute:contrail-vrouter-agent:0 sm_queue_count = 0 sm_stats= [ ev_stats= [ [ [ event = scm::EvCollectorUpdate enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvIdleHoldTimerExpired enqueues = 6 dequeues = 6 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvSandeshSend enqueues = 106 dequeues = 106 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvStart enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpConnectFail enqueues = 7 dequeues = 7 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpDeleteSession enqueues = 7 dequeues = 7 enqueue_fails = 0 dequeue_fails = 0 ], [ event = enqueues = 128 dequeues = 128 enqueue_fails = 0 dequeue_fails = 0 ], ] ] state = Idle state_since = 1756692206755379 last_state = Idle last_event = scm::EvSandeshSend last_event_at = 1756692231316546 ] max_sm_queue_count = 6 ] 2025-09-01 Mon 02:03:55:748.158 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848866952704, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:03:55:748.184 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848866952704, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:55:748.614 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848866952704, Pid 111888]: SANDESH: Send FAILED: 1756692235747820 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3992-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Idle successful_connections = 0 pid = 111888 http_port = 8085 start_time = 1756692145743123 collector_name = collector_ip = 10.0.0.254:8086 collector_list= [ [ (*_iter40) = 10.0.0.254:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ComputeCpuStateTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 19 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 20 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 30 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterControlStatsTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 3 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 56 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 15 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 48, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 106, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:03:55:748.677 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848866952704, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:03:55:753.616 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848854357504, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:55:754.167 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848854357504, Pid 111888]: SANDESH: Send FAILED: 1756692235753483 [SYS_NOTICE]: VrouterStats: data= [ name = cn-jenkins-deploy-platform-ansible-os-3992-1 in_tpkts = 51 in_bytes = 2254 out_tpkts = 61 out_bytes = 3110 exception_packets = 10 exception_packets_dropped = 10 raw_vhost_stats= [ name = vhost0 in_pkts = 51 in_bytes = 2254 out_pkts = 61 out_bytes = 3110 speed = -1 duplexity = -1 ] raw_vhost_drop_stats= [ ds_discard = 0 ds_pull = 0 ds_invalid_if = 0 ds_invalid_arp = 0 ds_trap_no_if = 0 ds_nowhere_to_go = 0 ds_flow_queue_limit_exceeded = 0 ds_flow_no_memory = 0 ds_flow_invalid_protocol = 0 ds_flow_nat_no_rflow = 0 ds_flow_action_drop = 0 ds_flow_action_invalid = 0 ds_flow_unusable = 0 ds_flow_table_full = 0 ds_interface_tx_discard = 0 ds_interface_drop = 0 ds_duplicated = 0 ds_push = 0 ds_ttl_exceeded = 0 ds_invalid_nh = 0 ds_invalid_label = 0 ds_invalid_protocol = 0 ds_interface_rx_discard = 0 ds_invalid_mcast_source = 0 ds_head_alloc_fail = 0 ds_pcow_fail = 0 ds_mcast_clone_fail = 0 ds_rewrite_fail = 0 ds_misc = 0 ds_invalid_packet = 0 ds_cksum_err = 0 ds_no_fmd = 0 ds_invalid_vnid = 0 ds_frag_err = 0 ds_invalid_source = 0 ds_mcast_df_bit = 0 ds_l2_no_route = 0 ds_vlan_fwd_tx = 0 ds_vlan_fwd_enq = 0 ds_drop_new_flow = 0 ds_flow_evict = 0 ds_trap_original = 0 ds_no_memory = 0 ds_fragment_queue_fail = 0 ds_no_frag_entry = 0 ds_icmp_error = 0 ds_clone_fail = 0 ds_drop_pkts = 0 ] raw_drop_stats= [ ds_discard = 0 ds_pull = 0 ds_invalid_if = 0 ds_invalid_arp = 0 ds_trap_no_if = 0 ds_nowhere_to_go = 0 ds_flow_queue_limit_exceeded = 0 ds_flow_no_memory = 0 ds_flow_invalid_protocol = 0 ds_flow_nat_no_rflow = 0 ds_flow_action_drop = 0 ds_flow_action_invalid = 0 ds_flow_unusable = 0 ds_flow_table_full = 0 ds_interface_tx_discard = 0 ds_interface_drop = 0 ds_duplicated = 0 ds_push = 0 ds_ttl_exceeded = 0 ds_invalid_nh = 0 ds_invalid_label = 0 ds_invalid_protocol = 0 ds_interface_rx_discard = 0 ds_invalid_mcast_source = 0 ds_head_alloc_fail = 0 ds_pcow_fail = 0 ds_mcast_clone_fail = 0 ds_rewrite_fail = 0 ds_misc = 0 ds_invalid_packet = 0 ds_cksum_err = 0 ds_no_fmd = 0 ds_invalid_vnid = 0 ds_frag_err = 0 ds_invalid_source = 0 ds_mcast_df_bit = 0 ds_l2_no_route = 0 ds_vlan_fwd_tx = 0 ds_vlan_fwd_enq = 0 ds_drop_new_flow = 0 ds_flow_evict = 0 ds_trap_original = 0 ds_no_memory = 0 ds_fragment_queue_fail = 0 ds_no_frag_entry = 0 ds_icmp_error = 0 ds_clone_fail = 0 ds_drop_pkts = 0 ] total_in_bandwidth_utilization = 7.69784e-16 total_out_bandwidth_utilization = 9.59519e-16 flow_rate= [ added_flows = 0 max_flow_adds_per_second = 0 min_flow_adds_per_second = 0 deleted_flows = 0 max_flow_deletes_per_second = 0 min_flow_deletes_per_second = 0 active_flows = 0 hold_flows = 0 ] phy_band_in_bps= [ [ _iter260->first = ens6 _iter260->second = 142, ] ] phy_band_out_bps= [ [ _iter261->first = ens6 _iter261->second = 177, ] ] raw_phy_if_stats= [ [ _iter262->first = ens6 [ in_pkts = 14 in_bytes = 1136 out_pkts = 51 out_bytes = 2254 ], ] ] raw_phy_if_drop_stats= [ [ _iter263->first = ens6 [ ds_discard = 0 ds_pull = 0 ds_invalid_if = 0 ds_invalid_arp = 0 ds_trap_no_if = 0 ds_nowhere_to_go = 0 ds_flow_queue_limit_exceeded = 0 ds_flow_no_memory = 0 ds_flow_invalid_protocol = 0 ds_flow_nat_no_rflow = 0 ds_flow_action_drop = 0 ds_flow_action_invalid = 0 ds_flow_unusable = 0 ds_flow_table_full = 0 ds_interface_tx_discard = 0 ds_interface_drop = 0 ds_duplicated = 0 ds_push = 0 ds_ttl_exceeded = 0 ds_invalid_nh = 0 ds_invalid_label = 0 ds_invalid_protocol = 0 ds_interface_rx_discard = 0 ds_invalid_mcast_source = 0 ds_head_alloc_fail = 0 ds_pcow_fail = 0 ds_mcast_clone_fail = 0 ds_rewrite_fail = 0 ds_misc = 0 ds_invalid_packet = 0 ds_cksum_err = 0 ds_no_fmd = 0 ds_invalid_vnid = 0 ds_frag_err = 0 ds_invalid_source = 0 ds_mcast_df_bit = 0 ds_l2_no_route = 0 ds_vlan_fwd_tx = 0 ds_vlan_fwd_enq = 0 ds_drop_new_flow = 0 ds_flow_evict = 0 ds_trap_original = 0 ds_no_memory = 0 ds_fragment_queue_fail = 0 ds_no_frag_entry = 0 ds_icmp_error = 0 ds_clone_fail = 0 ds_drop_pkts = 0 ], ] ] ] 2025-09-01 Mon 02:03:55:754.243 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848854357504, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: VrouterStats 2025-09-01 Mon 02:03:55:754.285 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848854357504, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:55:754.394 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848854357504, Pid 111888]: SANDESH: Send FAILED: 1756692235753694 [SYS_NOTICE]: VrouterControlStatsTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3992-1 raw_xmpp_stats= [ [ _iter306->first = 10.20.0.17 [ reconnects = 0 in_msgs = 0 out_msgs = 0 ], ] ] raw_ifmap_stats= [ [ _iter307->first = link_delete_parse_errors _iter307->second = 0, _iter307->first = link_update_parse_errors _iter307->second = 0, _iter307->first = node_delete_parse_errors _iter307->second = 0, _iter307->first = node_update_parse_errors _iter307->second = 0, ] ] raw_rt_table_size= [ [ _iter308->first = default-domain:default-project:ip-fabric:__default__ [ inet4_unicast = 4 inet4_multicast = 0 evpn = 0 bridge = 2 inet6_unicast = 2 ], _iter308->first = default-domain:default-project:ip-fabric:ip-fabric [ inet4_unicast = 2 inet4_multicast = 0 evpn = 1 bridge = 2 inet6_unicast = 0 ], ] ] ] 2025-09-01 Mon 02:03:55:754.440 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848854357504, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: VrouterControlStatsTrace 2025-09-01 Mon 02:03:56:755.535 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848862754304, Pid 111888]: Processing scm::EvIdleHoldTimerExpired in state Idle 2025-09-01 Mon 02:03:56:755.779 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848862754304, Pid 111888]: Connect : Start Connect timer 10.0.0.254:8086 2025-09-01 Mon 02:03:56:755.824 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848886536832, Pid 111888]: -(-1) OnSessionEvent TCP Connect Failed 2025-09-01 Mon 02:03:56:755.885 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848886536832, Pid 111888]: -(-1):EvTcpConnectFail 2025-09-01 Mon 02:03:56:755.890 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848862754304, Pid 111888]: Processing scm::EvSandeshSend in state Connect 2025-09-01 Mon 02:03:56:756.041 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848862754304, Pid 111888]: SANDESH: Send FAILED: 1756692236755607 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-3992-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.17 server_addrs= [ [ (*_iter6) = 10.20.0.17:5269, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.17 server_addrs= [ [ (*_iter6) = 10.20.0.17:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.254:8086, ] ] status = Initializing description = Idle : EvIdleHoldTimerExpired -> Connect ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.17, XMPP:dns-server:10.20.0.17, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:03:56:756.069 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848862754304, Pid 111888]: Wrong state: Connect for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:03:56:756.100 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848862754304, Pid 111888]: Processing scm::EvSandeshSend in state Connect 2025-09-01 Mon 02:03:56:756.472 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848862754304, Pid 111888]: SANDESH: Send FAILED: 1756692236755872 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3992-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Connect successful_connections = 0 pid = 111888 http_port = 8085 start_time = 1756692145743123 collector_name = collector_ip = 10.0.0.254:8086 collector_list= [ [ (*_iter40) = 10.0.0.254:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ComputeCpuStateTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 19 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 22 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 30 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterControlStatsTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 3 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 4 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 56 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 15 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 48, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 110, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:03:56:756.542 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848862754304, Pid 111888]: Wrong state: Connect for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:03:56:756.574 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848862754304, Pid 111888]: Processing scm::EvTcpConnectFail in state Connect 2025-09-01 Mon 02:03:56:756.604 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848862754304, Pid 111888]: Connect : EvTcpConnectFail 2025-09-01 Mon 02:03:56:756.756 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848862754304, Pid 111888]: TCP [SYS_ERR]: TcpSessionMessageLog: Session -(-1) > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:03:56:756.828 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848862754304, Pid 111888]: Idle 2025-09-01 Mon 02:03:56:757.001 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848862754304, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:56:757.057 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848862754304, Pid 111888]: SANDESH: Send FAILED: 1756692236756667 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-3992-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.17 server_addrs= [ [ (*_iter6) = 10.20.0.17:5269, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.17 server_addrs= [ [ (*_iter6) = 10.20.0.17:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.17, XMPP:dns-server:10.20.0.17, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:03:56:757.078 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848862754304, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:03:56:757.117 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848862754304, Pid 111888]: Processing scm::EvTcpDeleteSession in state Idle 2025-09-01 Mon 02:03:56:757.215 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848862754304, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:03:56:757.780 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848862754304, Pid 111888]: SANDESH: Send FAILED: 1756692236756966 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3992-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Idle successful_connections = 0 pid = 111888 http_port = 8085 start_time = 1756692145743123 collector_name = collector_ip = 10.0.0.254:8086 collector_list= [ [ (*_iter40) = 10.0.0.254:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ComputeCpuStateTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 20 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 23 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 31 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterControlStatsTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 3 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 4 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 56 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 15 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 49, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 112, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:03:56:757.845 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848862754304, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:03:58:536.243 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848886536832, Pid 111888]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.0.0.254:8085::10.0.0.254:56808 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 255 2025-09-01 Mon 02:04:01:117.314 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:01:117.446 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: SANDESH: Send FAILED: 1756692241117249 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.17 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:01:117.478 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:01:117.693 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848858555904, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:01:117.746 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848886536832, Pid 111888]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.17 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:04:01:117.772 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848858555904, Pid 111888]: SANDESH: Send FAILED: 1756692241117607 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.17 state = Connect event = Connect Event local_port = 33245 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:01:117.916 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848854357504, Pid 111888]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:04:01:118.005 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848858555904, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:01:118.104 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848858555904, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:01:118.148 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848858555904, Pid 111888]: SANDESH: Send FAILED: 1756692241118065 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.17 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:01:118.168 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848858555904, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:01:118.198 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848858555904, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:01:118.228 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848858555904, Pid 111888]: SANDESH: Send FAILED: 1756692241118073 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.17 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:01:118.252 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848858555904, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:11:510.382 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848886536832, Pid 111888]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.0.0.254:8085::10.0.0.254:46646 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 255 2025-09-01 Mon 02:04:19:667.003 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848862754304, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:19:667.137 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848862754304, Pid 111888]: SANDESH: Send FAILED: 1756692259666911 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.17 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:19:667.166 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848862754304, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:19:667.322 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848854357504, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:19:667.373 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848886536832, Pid 111888]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.17 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:04:19:667.436 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848854357504, Pid 111888]: SANDESH: Send FAILED: 1756692259667252 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.17 state = Connect event = Connect Event local_port = 40961 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:19:667.657 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848854357504, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:19:667.778 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:04:19:667.901 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848854357504, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:19:667.951 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848854357504, Pid 111888]: SANDESH: Send FAILED: 1756692259667881 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.17 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:19:667.981 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848854357504, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:19:668.013 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848854357504, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:19:668.043 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848854357504, Pid 111888]: SANDESH: Send FAILED: 1756692259667900 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.17 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:19:668.061 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848854357504, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:24:344.866 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848886536832, Pid 111888]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.0.0.254:8085::10.0.0.254:48250 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 255 2025-09-01 Mon 02:04:25:518.445 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848886536832, Pid 111888]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.20.0.17:8085::10.20.0.17:46592 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 255 2025-09-01 Mon 02:04:25:537.820 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848886536832, Pid 111888]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.20.0.17:8085::10.20.0.17:46604 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 255 2025-09-01 Mon 02:04:25:748.448 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848866952704, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:25:748.679 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848866952704, Pid 111888]: SANDESH: Send FAILED: 1756692265748150 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3992-1:Compute:contrail-vrouter-agent:0 sm_queue_count = 0 sm_stats= [ ev_stats= [ [ [ event = scm::EvCollectorUpdate enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvIdleHoldTimerExpired enqueues = 7 dequeues = 7 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvSandeshSend enqueues = 122 dequeues = 122 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvStart enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpConnectFail enqueues = 8 dequeues = 8 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpDeleteSession enqueues = 8 dequeues = 8 enqueue_fails = 0 dequeue_fails = 0 ], [ event = enqueues = 147 dequeues = 147 enqueue_fails = 0 dequeue_fails = 0 ], ] ] state = Idle state_since = 1756692236756824 last_state = Idle last_event = scm::EvSandeshSend last_event_at = 1756692259668009 ] max_sm_queue_count = 6 ] 2025-09-01 Mon 02:04:25:748.744 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848866952704, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:04:25:748.770 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848866952704, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:25:749.181 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848866952704, Pid 111888]: SANDESH: Send FAILED: 1756692265748348 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3992-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Idle successful_connections = 0 pid = 111888 http_port = 8085 start_time = 1756692145743123 collector_name = collector_ip = 10.0.0.254:8086 collector_list= [ [ (*_iter40) = 10.0.0.254:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ComputeCpuStateTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 21 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 24 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 38 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterControlStatsTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 3 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 4 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 64 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 17 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 58, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 122, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:04:25:749.227 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848866952704, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:04:25:754.050 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:25:754.263 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: SANDESH: Send FAILED: 1756692265753982 [SYS_NOTICE]: VrouterStats: data= [ name = cn-jenkins-deploy-platform-ansible-os-3992-1 cpu_info= [ num_cpu = 8 sys_mem_info= [ total = 32 used = 0 free = 510398136 buffers = 32561 cached = 510398136 node_type = ] meminfo= [ virt = 0 peakvirt = 61401632 res = 0 ] cpuload= [ one_min_avg = 0.21875 five_min_avg = 0.26625 fifteen_min_avg = 0.23875 ] cpu_share = 0.454167 ] ] 2025-09-01 Mon 02:04:25:754.314 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: VrouterStats 2025-09-01 Mon 02:04:25:754.356 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:25:754.472 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: SANDESH: Send FAILED: 1756692265754010 [SYS_NOTICE]: ComputeCpuStateTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3992-1 cpu_info= [ [ [ mem_virt = 0 cpu_share = 0.454167 used_sys_mem = 3784536599 one_min_cpuload = 0.21875 mem_res = 0 ], ] ] ] 2025-09-01 Mon 02:04:25:754.556 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: ComputeCpuStateTrace 2025-09-01 Mon 02:04:25:754.598 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:25:755.121 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: SANDESH: Send FAILED: 1756692265754096 [SYS_NOTICE]: VrouterStats: data= [ name = cn-jenkins-deploy-platform-ansible-os-3992-1 in_tpkts = 66 in_bytes = 2884 out_tpkts = 80 out_bytes = 4020 raw_vhost_stats= [ name = vhost0 in_pkts = 66 in_bytes = 2884 out_pkts = 80 out_bytes = 4020 speed = -1 duplexity = -1 ] raw_vhost_drop_stats= [ ds_discard = 0 ds_pull = 0 ds_invalid_if = 0 ds_invalid_arp = 0 ds_trap_no_if = 0 ds_nowhere_to_go = 0 ds_flow_queue_limit_exceeded = 0 ds_flow_no_memory = 0 ds_flow_invalid_protocol = 0 ds_flow_nat_no_rflow = 0 ds_flow_action_drop = 0 ds_flow_action_invalid = 0 ds_flow_unusable = 0 ds_flow_table_full = 0 ds_interface_tx_discard = 0 ds_interface_drop = 0 ds_duplicated = 0 ds_push = 0 ds_ttl_exceeded = 0 ds_invalid_nh = 0 ds_invalid_label = 0 ds_invalid_protocol = 0 ds_interface_rx_discard = 0 ds_invalid_mcast_source = 0 ds_head_alloc_fail = 0 ds_pcow_fail = 0 ds_mcast_clone_fail = 0 ds_rewrite_fail = 0 ds_misc = 0 ds_invalid_packet = 0 ds_cksum_err = 0 ds_no_fmd = 0 ds_invalid_vnid = 0 ds_frag_err = 0 ds_invalid_source = 0 ds_mcast_df_bit = 0 ds_l2_no_route = 0 ds_vlan_fwd_tx = 0 ds_vlan_fwd_enq = 0 ds_drop_new_flow = 0 ds_flow_evict = 0 ds_trap_original = 0 ds_no_memory = 0 ds_fragment_queue_fail = 0 ds_no_frag_entry = 0 ds_icmp_error = 0 ds_clone_fail = 0 ds_drop_pkts = 0 ] raw_drop_stats= [ ds_discard = 0 ds_pull = 0 ds_invalid_if = 0 ds_invalid_arp = 0 ds_trap_no_if = 0 ds_nowhere_to_go = 0 ds_flow_queue_limit_exceeded = 0 ds_flow_no_memory = 0 ds_flow_invalid_protocol = 0 ds_flow_nat_no_rflow = 0 ds_flow_action_drop = 0 ds_flow_action_invalid = 0 ds_flow_unusable = 0 ds_flow_table_full = 0 ds_interface_tx_discard = 0 ds_interface_drop = 0 ds_duplicated = 0 ds_push = 0 ds_ttl_exceeded = 0 ds_invalid_nh = 0 ds_invalid_label = 0 ds_invalid_protocol = 0 ds_interface_rx_discard = 0 ds_invalid_mcast_source = 0 ds_head_alloc_fail = 0 ds_pcow_fail = 0 ds_mcast_clone_fail = 0 ds_rewrite_fail = 0 ds_misc = 0 ds_invalid_packet = 0 ds_cksum_err = 0 ds_no_fmd = 0 ds_invalid_vnid = 0 ds_frag_err = 0 ds_invalid_source = 0 ds_mcast_df_bit = 0 ds_l2_no_route = 0 ds_vlan_fwd_tx = 0 ds_vlan_fwd_enq = 0 ds_drop_new_flow = 0 ds_flow_evict = 0 ds_trap_original = 0 ds_no_memory = 0 ds_fragment_queue_fail = 0 ds_no_frag_entry = 0 ds_icmp_error = 0 ds_clone_fail = 0 ds_drop_pkts = 0 ] flow_rate= [ added_flows = 0 max_flow_adds_per_second = 0 min_flow_adds_per_second = 0 deleted_flows = 0 max_flow_deletes_per_second = 0 min_flow_deletes_per_second = 0 active_flows = 0 hold_flows = 0 ] raw_phy_if_stats= [ [ _iter262->first = ens6 [ in_pkts = 18 in_bytes = 1416 out_pkts = 66 out_bytes = 2884 ], ] ] raw_phy_if_drop_stats= [ [ _iter263->first = ens6 [ ds_discard = 0 ds_pull = 0 ds_invalid_if = 0 ds_invalid_arp = 0 ds_trap_no_if = 0 ds_nowhere_to_go = 0 ds_flow_queue_limit_exceeded = 0 ds_flow_no_memory = 0 ds_flow_invalid_protocol = 0 ds_flow_nat_no_rflow = 0 ds_flow_action_drop = 0 ds_flow_action_invalid = 0 ds_flow_unusable = 0 ds_flow_table_full = 0 ds_interface_tx_discard = 0 ds_interface_drop = 0 ds_duplicated = 0 ds_push = 0 ds_ttl_exceeded = 0 ds_invalid_nh = 0 ds_invalid_label = 0 ds_invalid_protocol = 0 ds_interface_rx_discard = 0 ds_invalid_mcast_source = 0 ds_head_alloc_fail = 0 ds_pcow_fail = 0 ds_mcast_clone_fail = 0 ds_rewrite_fail = 0 ds_misc = 0 ds_invalid_packet = 0 ds_cksum_err = 0 ds_no_fmd = 0 ds_invalid_vnid = 0 ds_frag_err = 0 ds_invalid_source = 0 ds_mcast_df_bit = 0 ds_l2_no_route = 0 ds_vlan_fwd_tx = 0 ds_vlan_fwd_enq = 0 ds_drop_new_flow = 0 ds_flow_evict = 0 ds_trap_original = 0 ds_no_memory = 0 ds_fragment_queue_fail = 0 ds_no_frag_entry = 0 ds_icmp_error = 0 ds_clone_fail = 0 ds_drop_pkts = 0 ], ] ] ] 2025-09-01 Mon 02:04:25:755.189 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: VrouterStats 2025-09-01 Mon 02:04:25:755.221 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:25:755.296 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: SANDESH: Send FAILED: 1756692265754145 [SYS_NOTICE]: VrouterControlStatsTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3992-1 raw_xmpp_stats= [ [ _iter306->first = 10.20.0.17 [ reconnects = 0 in_msgs = 0 out_msgs = 0 ], ] ] raw_ifmap_stats= [ [ _iter307->first = link_delete_parse_errors _iter307->second = 0, _iter307->first = link_update_parse_errors _iter307->second = 0, _iter307->first = node_delete_parse_errors _iter307->second = 0, _iter307->first = node_update_parse_errors _iter307->second = 0, ] ] raw_rt_table_size= [ [ _iter308->first = default-domain:default-project:ip-fabric:__default__ [ inet4_unicast = 4 inet4_multicast = 0 evpn = 0 bridge = 2 inet6_unicast = 2 ], _iter308->first = default-domain:default-project:ip-fabric:ip-fabric [ inet4_unicast = 2 inet4_multicast = 0 evpn = 1 bridge = 2 inet6_unicast = 0 ], ] ] ] 2025-09-01 Mon 02:04:25:755.330 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: VrouterControlStatsTrace 2025-09-01 Mon 02:04:26:757.169 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848862754304, Pid 111888]: Processing scm::EvIdleHoldTimerExpired in state Idle 2025-09-01 Mon 02:04:26:757.827 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848862754304, Pid 111888]: Connect : Start Connect timer 10.0.0.254:8086 2025-09-01 Mon 02:04:26:757.875 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848886536832, Pid 111888]: -(-1) OnSessionEvent TCP Connect Failed 2025-09-01 Mon 02:04:26:757.958 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848886536832, Pid 111888]: -(-1):EvTcpConnectFail 2025-09-01 Mon 02:04:26:758.086 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848862754304, Pid 111888]: Processing scm::EvSandeshSend in state Connect 2025-09-01 Mon 02:04:26:758.159 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848862754304, Pid 111888]: SANDESH: Send FAILED: 1756692266757357 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-3992-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.17 server_addrs= [ [ (*_iter6) = 10.20.0.17:5269, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.17 server_addrs= [ [ (*_iter6) = 10.20.0.17:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.254:8086, ] ] status = Initializing description = Idle : EvIdleHoldTimerExpired -> Connect ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.17, XMPP:dns-server:10.20.0.17, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:04:26:758.184 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848862754304, Pid 111888]: Wrong state: Connect for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:04:26:758.212 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848862754304, Pid 111888]: Processing scm::EvTcpConnectFail in state Connect 2025-09-01 Mon 02:04:26:758.233 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848862754304, Pid 111888]: Connect : EvTcpConnectFail 2025-09-01 Mon 02:04:26:758.358 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848862754304, Pid 111888]: TCP [SYS_ERR]: TcpSessionMessageLog: Session -(-1) > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:04:26:758.448 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848862754304, Pid 111888]: Idle 2025-09-01 Mon 02:04:26:758.629 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848862754304, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:26:759.025 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848862754304, Pid 111888]: SANDESH: Send FAILED: 1756692266758039 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3992-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Connect successful_connections = 0 pid = 111888 http_port = 8085 start_time = 1756692145743123 collector_name = collector_ip = 10.0.0.254:8086 collector_list= [ [ (*_iter40) = 10.0.0.254:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ComputeCpuStateTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 21 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 26 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 38 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterControlStatsTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 4 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 6 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 64 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 17 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 58, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 128, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:04:26:759.097 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848862754304, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:04:26:759.123 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848862754304, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:26:759.173 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848862754304, Pid 111888]: SANDESH: Send FAILED: 1756692266758292 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-3992-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.17 server_addrs= [ [ (*_iter6) = 10.20.0.17:5269, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.17 server_addrs= [ [ (*_iter6) = 10.20.0.17:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.17, XMPP:dns-server:10.20.0.17, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:04:26:759.194 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848862754304, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:04:26:759.218 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848862754304, Pid 111888]: Processing scm::EvTcpDeleteSession in state Idle 2025-09-01 Mon 02:04:26:759.281 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848862754304, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:26:759.724 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848862754304, Pid 111888]: SANDESH: Send FAILED: 1756692266758594 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3992-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Idle successful_connections = 0 pid = 111888 http_port = 8085 start_time = 1756692145743123 collector_name = collector_ip = 10.0.0.254:8086 collector_list= [ [ (*_iter40) = 10.0.0.254:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ComputeCpuStateTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 22 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 26 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 39 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterControlStatsTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 4 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 6 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 64 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 17 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 59, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 129, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:04:26:759.777 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848862754304, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:04:30:008.390 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:30:008.586 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: SANDESH: Send FAILED: 1756692270008313 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.17 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:30:008.674 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:30:008.719 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:30:008.735 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848886536832, Pid 111888]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.17 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:04:30:008.839 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: SANDESH: Send FAILED: 1756692270008572 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.17 state = Connect event = Connect Event local_port = 44331 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:30:009.006 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:30:009.051 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848854357504, Pid 111888]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:04:30:009.150 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848866952704, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:30:009.236 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848866952704, Pid 111888]: SANDESH: Send FAILED: 1756692270009107 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.17 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:30:009.268 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848866952704, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:30:009.301 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848866952704, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:30:009.339 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848866952704, Pid 111888]: SANDESH: Send FAILED: 1756692270009116 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.17 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:30:009.362 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848866952704, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:37:169.252 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848886536832, Pid 111888]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.0.0.254:8085::10.0.0.254:51806 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 255 2025-09-01 Mon 02:04:48:018.243 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:48:018.378 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: SANDESH: Send FAILED: 1756692288018190 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.17 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:48:018.404 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:48:018.755 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848866952704, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:48:018.754 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848886536832, Pid 111888]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.17 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:04:48:018.893 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848866952704, Pid 111888]: SANDESH: Send FAILED: 1756692288018675 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.17 state = Connect event = Connect Event local_port = 43463 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:48:019.047 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848866952704, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:48:019.117 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848879547904, Pid 111888]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:04:48:019.349 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848866952704, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:48:019.385 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848866952704, Pid 111888]: SANDESH: Send FAILED: 1756692288019248 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.17 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:48:019.403 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848866952704, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:48:019.423 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848866952704, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:48:019.448 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848866952704, Pid 111888]: SANDESH: Send FAILED: 1756692288019302 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.17 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:04:48:019.465 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848866952704, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:04:50:402.045 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848886536832, Pid 111888]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.0.0.254:8085::10.0.0.254:40148 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 255 2025-09-01 Mon 02:04:55:748.829 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848862754304, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:55:748.931 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848862754304, Pid 111888]: SANDESH: Send FAILED: 1756692295748627 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3992-1:Compute:contrail-vrouter-agent:0 sm_queue_count = 0 sm_stats= [ ev_stats= [ [ [ event = scm::EvCollectorUpdate enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvIdleHoldTimerExpired enqueues = 8 dequeues = 8 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvSandeshSend enqueues = 140 dequeues = 140 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvStart enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpConnectFail enqueues = 9 dequeues = 9 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpDeleteSession enqueues = 9 dequeues = 9 enqueue_fails = 0 dequeue_fails = 0 ], [ event = enqueues = 168 dequeues = 168 enqueue_fails = 0 dequeue_fails = 0 ], ] ] state = Idle state_since = 1756692266758445 last_state = Idle last_event = scm::EvSandeshSend last_event_at = 1756692288019420 ] max_sm_queue_count = 6 ] 2025-09-01 Mon 02:04:55:748.947 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848862754304, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:04:55:748.960 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848862754304, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:55:749.135 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848862754304, Pid 111888]: SANDESH: Send FAILED: 1756692295748750 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3992-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Idle successful_connections = 0 pid = 111888 http_port = 8085 start_time = 1756692145743123 collector_name = collector_ip = 10.0.0.254:8086 collector_list= [ [ (*_iter40) = 10.0.0.254:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ComputeCpuStateTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 23 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 28 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 43 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterControlStatsTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 4 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 6 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 72 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 19 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 65, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 140, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:04:55:749.166 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848862754304, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:04:55:754.627 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848854357504, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:55:755.123 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848854357504, Pid 111888]: SANDESH: Send FAILED: 1756692295754501 [SYS_NOTICE]: VrouterStats: data= [ name = cn-jenkins-deploy-platform-ansible-os-3992-1 in_tpkts = 82 in_bytes = 3584 out_tpkts = 96 out_bytes = 4720 exception_packets = 11 exception_packets_dropped = 11 raw_vhost_stats= [ name = vhost0 in_pkts = 82 in_bytes = 3584 out_pkts = 96 out_bytes = 4720 speed = -1 duplexity = -1 ] raw_vhost_drop_stats= [ ds_discard = 0 ds_pull = 0 ds_invalid_if = 0 ds_invalid_arp = 0 ds_trap_no_if = 0 ds_nowhere_to_go = 0 ds_flow_queue_limit_exceeded = 0 ds_flow_no_memory = 0 ds_flow_invalid_protocol = 0 ds_flow_nat_no_rflow = 0 ds_flow_action_drop = 0 ds_flow_action_invalid = 0 ds_flow_unusable = 0 ds_flow_table_full = 0 ds_interface_tx_discard = 0 ds_interface_drop = 0 ds_duplicated = 0 ds_push = 0 ds_ttl_exceeded = 0 ds_invalid_nh = 0 ds_invalid_label = 0 ds_invalid_protocol = 0 ds_interface_rx_discard = 0 ds_invalid_mcast_source = 0 ds_head_alloc_fail = 0 ds_pcow_fail = 0 ds_mcast_clone_fail = 0 ds_rewrite_fail = 0 ds_misc = 0 ds_invalid_packet = 0 ds_cksum_err = 0 ds_no_fmd = 0 ds_invalid_vnid = 0 ds_frag_err = 0 ds_invalid_source = 0 ds_mcast_df_bit = 0 ds_l2_no_route = 0 ds_vlan_fwd_tx = 0 ds_vlan_fwd_enq = 0 ds_drop_new_flow = 0 ds_flow_evict = 0 ds_trap_original = 0 ds_no_memory = 0 ds_fragment_queue_fail = 0 ds_no_frag_entry = 0 ds_icmp_error = 0 ds_clone_fail = 0 ds_drop_pkts = 0 ] raw_drop_stats= [ ds_discard = 0 ds_pull = 0 ds_invalid_if = 0 ds_invalid_arp = 0 ds_trap_no_if = 0 ds_nowhere_to_go = 0 ds_flow_queue_limit_exceeded = 0 ds_flow_no_memory = 0 ds_flow_invalid_protocol = 0 ds_flow_nat_no_rflow = 0 ds_flow_action_drop = 0 ds_flow_action_invalid = 0 ds_flow_unusable = 0 ds_flow_table_full = 0 ds_interface_tx_discard = 0 ds_interface_drop = 0 ds_duplicated = 0 ds_push = 0 ds_ttl_exceeded = 0 ds_invalid_nh = 0 ds_invalid_label = 0 ds_invalid_protocol = 0 ds_interface_rx_discard = 0 ds_invalid_mcast_source = 0 ds_head_alloc_fail = 0 ds_pcow_fail = 0 ds_mcast_clone_fail = 0 ds_rewrite_fail = 0 ds_misc = 0 ds_invalid_packet = 0 ds_cksum_err = 0 ds_no_fmd = 0 ds_invalid_vnid = 0 ds_frag_err = 0 ds_invalid_source = 0 ds_mcast_df_bit = 0 ds_l2_no_route = 0 ds_vlan_fwd_tx = 0 ds_vlan_fwd_enq = 0 ds_drop_new_flow = 0 ds_flow_evict = 0 ds_trap_original = 0 ds_no_memory = 0 ds_fragment_queue_fail = 0 ds_no_frag_entry = 0 ds_icmp_error = 0 ds_clone_fail = 0 ds_drop_pkts = 0 ] total_in_bandwidth_utilization = 2.49366e-16 flow_rate= [ added_flows = 0 max_flow_adds_per_second = 0 min_flow_adds_per_second = 0 deleted_flows = 0 max_flow_deletes_per_second = 0 min_flow_deletes_per_second = 0 active_flows = 0 hold_flows = 0 ] phy_band_in_bps= [ [ _iter260->first = ens6 _iter260->second = 46, ] ] phy_band_out_bps= [ [ _iter261->first = ens6 _iter261->second = 177, ] ] raw_phy_if_stats= [ [ _iter262->first = ens6 [ in_pkts = 19 in_bytes = 1486 out_pkts = 82 out_bytes = 3584 ], ] ] raw_phy_if_drop_stats= [ [ _iter263->first = ens6 [ ds_discard = 0 ds_pull = 0 ds_invalid_if = 0 ds_invalid_arp = 0 ds_trap_no_if = 0 ds_nowhere_to_go = 0 ds_flow_queue_limit_exceeded = 0 ds_flow_no_memory = 0 ds_flow_invalid_protocol = 0 ds_flow_nat_no_rflow = 0 ds_flow_action_drop = 0 ds_flow_action_invalid = 0 ds_flow_unusable = 0 ds_flow_table_full = 0 ds_interface_tx_discard = 0 ds_interface_drop = 0 ds_duplicated = 0 ds_push = 0 ds_ttl_exceeded = 0 ds_invalid_nh = 0 ds_invalid_label = 0 ds_invalid_protocol = 0 ds_interface_rx_discard = 0 ds_invalid_mcast_source = 0 ds_head_alloc_fail = 0 ds_pcow_fail = 0 ds_mcast_clone_fail = 0 ds_rewrite_fail = 0 ds_misc = 0 ds_invalid_packet = 0 ds_cksum_err = 0 ds_no_fmd = 0 ds_invalid_vnid = 0 ds_frag_err = 0 ds_invalid_source = 0 ds_mcast_df_bit = 0 ds_l2_no_route = 0 ds_vlan_fwd_tx = 0 ds_vlan_fwd_enq = 0 ds_drop_new_flow = 0 ds_flow_evict = 0 ds_trap_original = 0 ds_no_memory = 0 ds_fragment_queue_fail = 0 ds_no_frag_entry = 0 ds_icmp_error = 0 ds_clone_fail = 0 ds_drop_pkts = 0 ], ] ] ] 2025-09-01 Mon 02:04:55:755.192 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848854357504, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: VrouterStats 2025-09-01 Mon 02:04:55:755.220 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848854357504, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:55:755.354 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848854357504, Pid 111888]: SANDESH: Send FAILED: 1756692295754660 [SYS_NOTICE]: VrouterControlStatsTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3992-1 raw_xmpp_stats= [ [ _iter306->first = 10.20.0.17 [ reconnects = 0 in_msgs = 0 out_msgs = 0 ], ] ] raw_ifmap_stats= [ [ _iter307->first = link_delete_parse_errors _iter307->second = 0, _iter307->first = link_update_parse_errors _iter307->second = 0, _iter307->first = node_delete_parse_errors _iter307->second = 0, _iter307->first = node_update_parse_errors _iter307->second = 0, ] ] raw_rt_table_size= [ [ _iter308->first = default-domain:default-project:ip-fabric:__default__ [ inet4_unicast = 4 inet4_multicast = 0 evpn = 0 bridge = 2 inet6_unicast = 2 ], _iter308->first = default-domain:default-project:ip-fabric:ip-fabric [ inet4_unicast = 2 inet4_multicast = 0 evpn = 1 bridge = 2 inet6_unicast = 0 ], ] ] ] 2025-09-01 Mon 02:04:55:755.375 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848854357504, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: VrouterControlStatsTrace 2025-09-01 Mon 02:04:56:758.693 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848866952704, Pid 111888]: Processing scm::EvIdleHoldTimerExpired in state Idle 2025-09-01 Mon 02:04:56:759.052 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848866952704, Pid 111888]: Connect : Start Connect timer 10.0.0.254:8086 2025-09-01 Mon 02:04:56:759.136 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848886536832, Pid 111888]: -(-1) OnSessionEvent TCP Connect Failed 2025-09-01 Mon 02:04:56:759.205 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848866952704, Pid 111888]: Processing scm::EvSandeshSend in state Connect 2025-09-01 Mon 02:04:56:759.224 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848886536832, Pid 111888]: -(-1):EvTcpConnectFail 2025-09-01 Mon 02:04:56:759.269 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848866952704, Pid 111888]: SANDESH: Send FAILED: 1756692296758826 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-3992-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.17 server_addrs= [ [ (*_iter6) = 10.20.0.17:5269, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.17 server_addrs= [ [ (*_iter6) = 10.20.0.17:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.254:8086, ] ] status = Initializing description = Idle : EvIdleHoldTimerExpired -> Connect ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.17, XMPP:dns-server:10.20.0.17, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:04:56:759.338 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848866952704, Pid 111888]: Wrong state: Connect for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:04:56:759.351 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848866952704, Pid 111888]: Processing scm::EvSandeshSend in state Connect 2025-09-01 Mon 02:04:56:759.584 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848866952704, Pid 111888]: SANDESH: Send FAILED: 1756692296759179 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3992-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Connect successful_connections = 0 pid = 111888 http_port = 8085 start_time = 1756692145743123 collector_name = collector_ip = 10.0.0.254:8086 collector_list= [ [ (*_iter40) = 10.0.0.254:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ComputeCpuStateTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 23 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 30 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 43 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterControlStatsTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 5 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 7 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 72 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 19 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 65, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 144, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:04:56:759.622 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848866952704, Pid 111888]: Wrong state: Connect for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:04:56:759.639 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848866952704, Pid 111888]: Processing scm::EvTcpConnectFail in state Connect 2025-09-01 Mon 02:04:56:759.648 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848866952704, Pid 111888]: Connect : EvTcpConnectFail 2025-09-01 Mon 02:04:56:759.706 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848866952704, Pid 111888]: TCP [SYS_ERR]: TcpSessionMessageLog: Session -(-1) > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:04:56:759.746 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848866952704, Pid 111888]: Idle 2025-09-01 Mon 02:04:56:759.830 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848866952704, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:56:759.857 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848866952704, Pid 111888]: SANDESH: Send FAILED: 1756692296759676 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-3992-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.17 server_addrs= [ [ (*_iter6) = 10.20.0.17:5269, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.17 server_addrs= [ [ (*_iter6) = 10.20.0.17:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.17, XMPP:dns-server:10.20.0.17, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:04:56:759.867 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848866952704, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:04:56:759.877 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848866952704, Pid 111888]: Processing scm::EvTcpDeleteSession in state Idle 2025-09-01 Mon 02:04:56:759.950 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848866952704, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:04:56:760.179 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848866952704, Pid 111888]: SANDESH: Send FAILED: 1756692296759812 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3992-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Idle successful_connections = 0 pid = 111888 http_port = 8085 start_time = 1756692145743123 collector_name = collector_ip = 10.0.0.254:8086 collector_list= [ [ (*_iter40) = 10.0.0.254:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ComputeCpuStateTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 24 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 31 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 44 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterControlStatsTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 5 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 7 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 72 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 19 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 66, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 146, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:04:56:760.206 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848866952704, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:05:01:059.462 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848862754304, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:01:059.579 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848862754304, Pid 111888]: SANDESH: Send FAILED: 1756692301059403 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.17 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:01:059.621 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848862754304, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:01:060.026 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848886536832, Pid 111888]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.17 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:05:01:060.063 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848858555904, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:01:060.131 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848862754304, Pid 111888]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:05:01:060.153 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848858555904, Pid 111888]: SANDESH: Send FAILED: 1756692301059996 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.17 state = Connect event = Connect Event local_port = 44543 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:01:060.183 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848858555904, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:01:060.210 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848858555904, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:01:060.246 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848858555904, Pid 111888]: SANDESH: Send FAILED: 1756692301060186 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.17 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:01:060.302 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848858555904, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:01:060.332 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848858555904, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:01:060.363 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848858555904, Pid 111888]: SANDESH: Send FAILED: 1756692301060195 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.17 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:01:060.387 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848858555904, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:03:241.693 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848886536832, Pid 111888]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.0.0.254:8085::10.0.0.254:56166 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 255 2025-09-01 Mon 02:05:16:080.004 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848886536832, Pid 111888]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.0.0.254:8085::10.0.0.254:53262 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 255 2025-09-01 Mon 02:05:20:149.783 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:20:149.932 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: SANDESH: Send FAILED: 1756692320149726 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.17 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:20:149.961 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:20:150.122 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848886536832, Pid 111888]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.17 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:05:20:150.403 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848854357504, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:20:151.377 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848854357504, Pid 111888]: SANDESH: Send FAILED: 1756692320150312 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.17 state = Connect event = Connect Event local_port = 44089 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:20:151.394 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848854357504, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:20:150.421 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848879547904, Pid 111888]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:05:20:151.674 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848854357504, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:20:151.688 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848854357504, Pid 111888]: SANDESH: Send FAILED: 1756692320151595 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.17 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:20:151.694 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848854357504, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:20:151.701 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848854357504, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:20:151.710 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848854357504, Pid 111888]: SANDESH: Send FAILED: 1756692320151690 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.17 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:20:151.715 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848854357504, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:25:749.117 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:25:749.198 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: SANDESH: Send FAILED: 1756692325748947 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3992-1:Compute:contrail-vrouter-agent:0 sm_queue_count = 0 sm_stats= [ ev_stats= [ [ [ event = scm::EvCollectorUpdate enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvIdleHoldTimerExpired enqueues = 9 dequeues = 9 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvSandeshSend enqueues = 156 dequeues = 156 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvStart enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpConnectFail enqueues = 10 dequeues = 10 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpDeleteSession enqueues = 10 dequeues = 10 enqueue_fails = 0 dequeue_fails = 0 ], [ event = enqueues = 187 dequeues = 187 enqueue_fails = 0 dequeue_fails = 0 ], ] ] state = Idle state_since = 1756692296759745 last_state = Idle last_event = scm::EvSandeshSend last_event_at = 1756692320151700 ] max_sm_queue_count = 6 ] 2025-09-01 Mon 02:05:25:749.211 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:05:25:749.220 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:25:749.347 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: SANDESH: Send FAILED: 1756692325749055 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3992-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Idle successful_connections = 0 pid = 111888 http_port = 8085 start_time = 1756692145743123 collector_name = collector_ip = 10.0.0.254:8086 collector_list= [ [ (*_iter40) = 10.0.0.254:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ComputeCpuStateTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 25 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 32 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 48 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterControlStatsTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 5 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 7 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 80 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 21 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 72, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 156, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:05:25:749.368 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:05:25:755.108 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848879547904, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:25:755.206 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848879547904, Pid 111888]: SANDESH: Send FAILED: 1756692325755040 [SYS_NOTICE]: VrouterStats: data= [ name = cn-jenkins-deploy-platform-ansible-os-3992-1 cpu_info= [ num_cpu = 8 sys_mem_info= [ total = 32 used = 0 free = 0 buffers = 0 cached = 510398136 node_type = ] meminfo= [ virt = 32561 peakvirt = 0 res = 0 ] cpuload= [ one_min_avg = 0.15625 five_min_avg = 0.24 fifteen_min_avg = 0.2325 ] cpu_share = 0.44375 ] ] 2025-09-01 Mon 02:05:25:755.221 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848879547904, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: VrouterStats 2025-09-01 Mon 02:05:25:755.235 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848879547904, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:25:755.258 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848879547904, Pid 111888]: SANDESH: Send FAILED: 1756692325755066 [SYS_NOTICE]: ComputeCpuStateTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3992-1 cpu_info= [ [ [ mem_virt = 32561 cpu_share = 0.44375 used_sys_mem = 3784569160 one_min_cpuload = 0.15625 mem_res = 0 ], ] ] ] 2025-09-01 Mon 02:05:25:755.268 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848879547904, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: ComputeCpuStateTrace 2025-09-01 Mon 02:05:25:755.303 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848879547904, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:25:755.488 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848879547904, Pid 111888]: SANDESH: Send FAILED: 1756692325755089 [SYS_NOTICE]: VrouterStats: data= [ name = cn-jenkins-deploy-platform-ansible-os-3992-1 in_tpkts = 97 in_bytes = 4214 out_tpkts = 111 out_bytes = 5350 raw_vhost_stats= [ name = vhost0 in_pkts = 97 in_bytes = 4214 out_pkts = 111 out_bytes = 5350 speed = -1 duplexity = -1 ] raw_vhost_drop_stats= [ ds_discard = 0 ds_pull = 0 ds_invalid_if = 0 ds_invalid_arp = 0 ds_trap_no_if = 0 ds_nowhere_to_go = 0 ds_flow_queue_limit_exceeded = 0 ds_flow_no_memory = 0 ds_flow_invalid_protocol = 0 ds_flow_nat_no_rflow = 0 ds_flow_action_drop = 0 ds_flow_action_invalid = 0 ds_flow_unusable = 0 ds_flow_table_full = 0 ds_interface_tx_discard = 0 ds_interface_drop = 0 ds_duplicated = 0 ds_push = 0 ds_ttl_exceeded = 0 ds_invalid_nh = 0 ds_invalid_label = 0 ds_invalid_protocol = 0 ds_interface_rx_discard = 0 ds_invalid_mcast_source = 0 ds_head_alloc_fail = 0 ds_pcow_fail = 0 ds_mcast_clone_fail = 0 ds_rewrite_fail = 0 ds_misc = 0 ds_invalid_packet = 0 ds_cksum_err = 0 ds_no_fmd = 0 ds_invalid_vnid = 0 ds_frag_err = 0 ds_invalid_source = 0 ds_mcast_df_bit = 0 ds_l2_no_route = 0 ds_vlan_fwd_tx = 0 ds_vlan_fwd_enq = 0 ds_drop_new_flow = 0 ds_flow_evict = 0 ds_trap_original = 0 ds_no_memory = 0 ds_fragment_queue_fail = 0 ds_no_frag_entry = 0 ds_icmp_error = 0 ds_clone_fail = 0 ds_drop_pkts = 0 ] raw_drop_stats= [ ds_discard = 0 ds_pull = 0 ds_invalid_if = 0 ds_invalid_arp = 0 ds_trap_no_if = 0 ds_nowhere_to_go = 0 ds_flow_queue_limit_exceeded = 0 ds_flow_no_memory = 0 ds_flow_invalid_protocol = 0 ds_flow_nat_no_rflow = 0 ds_flow_action_drop = 0 ds_flow_action_invalid = 0 ds_flow_unusable = 0 ds_flow_table_full = 0 ds_interface_tx_discard = 0 ds_interface_drop = 0 ds_duplicated = 0 ds_push = 0 ds_ttl_exceeded = 0 ds_invalid_nh = 0 ds_invalid_label = 0 ds_invalid_protocol = 0 ds_interface_rx_discard = 0 ds_invalid_mcast_source = 0 ds_head_alloc_fail = 0 ds_pcow_fail = 0 ds_mcast_clone_fail = 0 ds_rewrite_fail = 0 ds_misc = 0 ds_invalid_packet = 0 ds_cksum_err = 0 ds_no_fmd = 0 ds_invalid_vnid = 0 ds_frag_err = 0 ds_invalid_source = 0 ds_mcast_df_bit = 0 ds_l2_no_route = 0 ds_vlan_fwd_tx = 0 ds_vlan_fwd_enq = 0 ds_drop_new_flow = 0 ds_flow_evict = 0 ds_trap_original = 0 ds_no_memory = 0 ds_fragment_queue_fail = 0 ds_no_frag_entry = 0 ds_icmp_error = 0 ds_clone_fail = 0 ds_drop_pkts = 0 ] flow_rate= [ added_flows = 0 max_flow_adds_per_second = 0 min_flow_adds_per_second = 0 deleted_flows = 0 max_flow_deletes_per_second = 0 min_flow_deletes_per_second = 0 active_flows = 0 hold_flows = 0 ] raw_phy_if_stats= [ [ _iter262->first = ens6 [ in_pkts = 19 in_bytes = 1486 out_pkts = 97 out_bytes = 4214 ], ] ] raw_phy_if_drop_stats= [ [ _iter263->first = ens6 [ ds_discard = 0 ds_pull = 0 ds_invalid_if = 0 ds_invalid_arp = 0 ds_trap_no_if = 0 ds_nowhere_to_go = 0 ds_flow_queue_limit_exceeded = 0 ds_flow_no_memory = 0 ds_flow_invalid_protocol = 0 ds_flow_nat_no_rflow = 0 ds_flow_action_drop = 0 ds_flow_action_invalid = 0 ds_flow_unusable = 0 ds_flow_table_full = 0 ds_interface_tx_discard = 0 ds_interface_drop = 0 ds_duplicated = 0 ds_push = 0 ds_ttl_exceeded = 0 ds_invalid_nh = 0 ds_invalid_label = 0 ds_invalid_protocol = 0 ds_interface_rx_discard = 0 ds_invalid_mcast_source = 0 ds_head_alloc_fail = 0 ds_pcow_fail = 0 ds_mcast_clone_fail = 0 ds_rewrite_fail = 0 ds_misc = 0 ds_invalid_packet = 0 ds_cksum_err = 0 ds_no_fmd = 0 ds_invalid_vnid = 0 ds_frag_err = 0 ds_invalid_source = 0 ds_mcast_df_bit = 0 ds_l2_no_route = 0 ds_vlan_fwd_tx = 0 ds_vlan_fwd_enq = 0 ds_drop_new_flow = 0 ds_flow_evict = 0 ds_trap_original = 0 ds_no_memory = 0 ds_fragment_queue_fail = 0 ds_no_frag_entry = 0 ds_icmp_error = 0 ds_clone_fail = 0 ds_drop_pkts = 0 ], ] ] ] 2025-09-01 Mon 02:05:25:755.508 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848879547904, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: VrouterStats 2025-09-01 Mon 02:05:25:755.538 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848879547904, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:25:755.576 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848879547904, Pid 111888]: SANDESH: Send FAILED: 1756692325755121 [SYS_NOTICE]: VrouterControlStatsTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3992-1 raw_xmpp_stats= [ [ _iter306->first = 10.20.0.17 [ reconnects = 0 in_msgs = 0 out_msgs = 0 ], ] ] raw_ifmap_stats= [ [ _iter307->first = link_delete_parse_errors _iter307->second = 0, _iter307->first = link_update_parse_errors _iter307->second = 0, _iter307->first = node_delete_parse_errors _iter307->second = 0, _iter307->first = node_update_parse_errors _iter307->second = 0, ] ] raw_rt_table_size= [ [ _iter308->first = default-domain:default-project:ip-fabric:__default__ [ inet4_unicast = 4 inet4_multicast = 0 evpn = 0 bridge = 2 inet6_unicast = 2 ], _iter308->first = default-domain:default-project:ip-fabric:ip-fabric [ inet4_unicast = 2 inet4_multicast = 0 evpn = 1 bridge = 2 inet6_unicast = 0 ], ] ] ] 2025-09-01 Mon 02:05:25:755.585 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848879547904, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: VrouterControlStatsTrace 2025-09-01 Mon 02:05:26:610.824 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848886536832, Pid 111888]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.20.0.17:8085::10.20.0.17:54570 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 255 2025-09-01 Mon 02:05:26:633.725 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848886536832, Pid 111888]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.20.0.17:8085::10.20.0.17:54578 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 255 2025-09-01 Mon 02:05:26:760.257 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848862754304, Pid 111888]: Processing scm::EvIdleHoldTimerExpired in state Idle 2025-09-01 Mon 02:05:26:760.751 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848862754304, Pid 111888]: Connect : Start Connect timer 10.0.0.254:8086 2025-09-01 Mon 02:05:26:760.987 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848862754304, Pid 111888]: Processing scm::EvSandeshSend in state Connect 2025-09-01 Mon 02:05:26:761.056 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848862754304, Pid 111888]: SANDESH: Send FAILED: 1756692326760426 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-3992-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.17 server_addrs= [ [ (*_iter6) = 10.20.0.17:5269, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.17 server_addrs= [ [ (*_iter6) = 10.20.0.17:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.254:8086, ] ] status = Initializing description = Idle : EvIdleHoldTimerExpired -> Connect ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.17, XMPP:dns-server:10.20.0.17, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:05:26:761.084 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848862754304, Pid 111888]: Wrong state: Connect for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:05:26:761.136 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848862754304, Pid 111888]: Processing scm::EvSandeshSend in state Connect 2025-09-01 Mon 02:05:26:761.093 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848886536832, Pid 111888]: -(-1) OnSessionEvent TCP Connect Failed 2025-09-01 Mon 02:05:26:761.301 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848886536832, Pid 111888]: -(-1):EvTcpConnectFail 2025-09-01 Mon 02:05:26:761.628 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848862754304, Pid 111888]: SANDESH: Send FAILED: 1756692326760937 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3992-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Connect successful_connections = 0 pid = 111888 http_port = 8085 start_time = 1756692145743123 collector_name = collector_ip = 10.0.0.254:8086 collector_list= [ [ (*_iter40) = 10.0.0.254:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ComputeCpuStateTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 3 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 25 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 34 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 50 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterControlStatsTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 6 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 9 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 80 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 21 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 74, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 162, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:05:26:761.736 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848862754304, Pid 111888]: Wrong state: Connect for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:05:26:761.767 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848862754304, Pid 111888]: Processing scm::EvTcpConnectFail in state Connect 2025-09-01 Mon 02:05:26:761.788 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848862754304, Pid 111888]: Connect : EvTcpConnectFail 2025-09-01 Mon 02:05:26:761.902 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848862754304, Pid 111888]: TCP [SYS_ERR]: TcpSessionMessageLog: Session -(-1) > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:05:26:761.972 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848862754304, Pid 111888]: Idle 2025-09-01 Mon 02:05:26:762.129 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848862754304, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:26:762.224 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848862754304, Pid 111888]: SANDESH: Send FAILED: 1756692326761847 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-3992-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.17 server_addrs= [ [ (*_iter6) = 10.20.0.17:5269, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.17 server_addrs= [ [ (*_iter6) = 10.20.0.17:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.17, XMPP:dns-server:10.20.0.17, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:05:26:762.257 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848862754304, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:05:26:762.292 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848862754304, Pid 111888]: Processing scm::EvTcpDeleteSession in state Idle 2025-09-01 Mon 02:05:26:762.377 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848862754304, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:26:762.822 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848862754304, Pid 111888]: SANDESH: Send FAILED: 1756692326762088 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3992-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Idle successful_connections = 0 pid = 111888 http_port = 8085 start_time = 1756692145743123 collector_name = collector_ip = 10.0.0.254:8086 collector_list= [ [ (*_iter40) = 10.0.0.254:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ComputeCpuStateTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 3 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 26 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 35 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 51 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterControlStatsTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 6 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 9 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 80 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 21 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 75, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 164, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:05:26:762.905 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848862754304, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:05:29:027.901 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848886536832, Pid 111888]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.0.0.254:8085::10.0.0.254:35058 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 255 2025-09-01 Mon 02:05:30:220.408 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848858555904, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:30:220.485 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848858555904, Pid 111888]: SANDESH: Send FAILED: 1756692330220369 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.17 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:30:220.502 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848858555904, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:30:220.608 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848854357504, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:30:220.615 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848886536832, Pid 111888]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.17 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:05:30:220.671 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848854357504, Pid 111888]: SANDESH: Send FAILED: 1756692330220575 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.17 state = Connect event = Connect Event local_port = 34415 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:30:220.719 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848854357504, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:30:220.735 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848858555904, Pid 111888]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:05:30:220.834 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848879547904, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:30:220.884 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848879547904, Pid 111888]: SANDESH: Send FAILED: 1756692330220773 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.17 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:30:220.897 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848879547904, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:30:220.911 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848879547904, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:30:220.928 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848879547904, Pid 111888]: SANDESH: Send FAILED: 1756692330220795 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.17 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:30:220.938 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848879547904, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:41:925.554 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848886536832, Pid 111888]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.0.0.254:8085::10.0.0.254:32816 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 255 2025-09-01 Mon 02:05:48:232.212 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848854357504, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:48:232.317 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848854357504, Pid 111888]: SANDESH: Send FAILED: 1756692348231923 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.17 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:48:232.345 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848854357504, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:48:232.370 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848854357504, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:48:232.406 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848854357504, Pid 111888]: SANDESH: Send FAILED: 1756692348232171 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.17 state = Connect event = Connect Event local_port = 43535 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:48:232.424 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848854357504, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:48:232.505 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848886536832, Pid 111888]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.17 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:05:48:232.663 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848858555904, Pid 111888]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:05:48:232.779 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:48:232.877 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: SANDESH: Send FAILED: 1756692348232718 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.17 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:48:232.904 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:48:232.929 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:48:232.960 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: SANDESH: Send FAILED: 1756692348232729 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.17 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:48:232.978 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:55:155.944 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848886536832, Pid 111888]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.0.0.254:8085::10.0.0.254:60778 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 255 2025-09-01 Mon 02:05:55:749.348 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848866952704, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:55:749.439 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848866952704, Pid 111888]: SANDESH: Send FAILED: 1756692355749195 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3992-1:Compute:contrail-vrouter-agent:0 sm_queue_count = 0 sm_stats= [ ev_stats= [ [ [ event = scm::EvCollectorUpdate enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvIdleHoldTimerExpired enqueues = 10 dequeues = 10 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvSandeshSend enqueues = 174 dequeues = 174 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvStart enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpConnectFail enqueues = 11 dequeues = 11 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpDeleteSession enqueues = 11 dequeues = 11 enqueue_fails = 0 dequeue_fails = 0 ], [ event = enqueues = 208 dequeues = 208 enqueue_fails = 0 dequeue_fails = 0 ], ] ] state = Idle state_since = 1756692326761969 last_state = Idle last_event = scm::EvSandeshSend last_event_at = 1756692348232925 ] max_sm_queue_count = 6 ] 2025-09-01 Mon 02:05:55:749.451 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848866952704, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:05:55:749.459 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848866952704, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:55:749.593 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848866952704, Pid 111888]: SANDESH: Send FAILED: 1756692355749310 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3992-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Idle successful_connections = 0 pid = 111888 http_port = 8085 start_time = 1756692145743123 collector_name = collector_ip = 10.0.0.254:8086 collector_list= [ [ (*_iter40) = 10.0.0.254:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ComputeCpuStateTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 3 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 27 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 36 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 56 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterControlStatsTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 6 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 9 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 88 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 23 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 82, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 174, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:05:55:749.610 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848866952704, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:05:55:755.305 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848879547904, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:55:755.441 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848879547904, Pid 111888]: SANDESH: Send FAILED: 1756692355755277 [SYS_NOTICE]: VrouterStats: data= [ name = cn-jenkins-deploy-platform-ansible-os-3992-1 in_tpkts = 112 in_bytes = 4844 out_tpkts = 127 out_bytes = 6050 raw_vhost_stats= [ name = vhost0 in_pkts = 112 in_bytes = 4844 out_pkts = 127 out_bytes = 6050 speed = -1 duplexity = -1 ] raw_vhost_drop_stats= [ ds_discard = 0 ds_pull = 0 ds_invalid_if = 0 ds_invalid_arp = 0 ds_trap_no_if = 0 ds_nowhere_to_go = 0 ds_flow_queue_limit_exceeded = 0 ds_flow_no_memory = 0 ds_flow_invalid_protocol = 0 ds_flow_nat_no_rflow = 0 ds_flow_action_drop = 0 ds_flow_action_invalid = 0 ds_flow_unusable = 0 ds_flow_table_full = 0 ds_interface_tx_discard = 0 ds_interface_drop = 0 ds_duplicated = 0 ds_push = 0 ds_ttl_exceeded = 0 ds_invalid_nh = 0 ds_invalid_label = 0 ds_invalid_protocol = 0 ds_interface_rx_discard = 0 ds_invalid_mcast_source = 0 ds_head_alloc_fail = 0 ds_pcow_fail = 0 ds_mcast_clone_fail = 0 ds_rewrite_fail = 0 ds_misc = 0 ds_invalid_packet = 0 ds_cksum_err = 0 ds_no_fmd = 0 ds_invalid_vnid = 0 ds_frag_err = 0 ds_invalid_source = 0 ds_mcast_df_bit = 0 ds_l2_no_route = 0 ds_vlan_fwd_tx = 0 ds_vlan_fwd_enq = 0 ds_drop_new_flow = 0 ds_flow_evict = 0 ds_trap_original = 0 ds_no_memory = 0 ds_fragment_queue_fail = 0 ds_no_frag_entry = 0 ds_icmp_error = 0 ds_clone_fail = 0 ds_drop_pkts = 0 ] raw_drop_stats= [ ds_discard = 0 ds_pull = 0 ds_invalid_if = 0 ds_invalid_arp = 0 ds_trap_no_if = 0 ds_nowhere_to_go = 0 ds_flow_queue_limit_exceeded = 0 ds_flow_no_memory = 0 ds_flow_invalid_protocol = 0 ds_flow_nat_no_rflow = 0 ds_flow_action_drop = 0 ds_flow_action_invalid = 0 ds_flow_unusable = 0 ds_flow_table_full = 0 ds_interface_tx_discard = 0 ds_interface_drop = 0 ds_duplicated = 0 ds_push = 0 ds_ttl_exceeded = 0 ds_invalid_nh = 0 ds_invalid_label = 0 ds_invalid_protocol = 0 ds_interface_rx_discard = 0 ds_invalid_mcast_source = 0 ds_head_alloc_fail = 0 ds_pcow_fail = 0 ds_mcast_clone_fail = 0 ds_rewrite_fail = 0 ds_misc = 0 ds_invalid_packet = 0 ds_cksum_err = 0 ds_no_fmd = 0 ds_invalid_vnid = 0 ds_frag_err = 0 ds_invalid_source = 0 ds_mcast_df_bit = 0 ds_l2_no_route = 0 ds_vlan_fwd_tx = 0 ds_vlan_fwd_enq = 0 ds_drop_new_flow = 0 ds_flow_evict = 0 ds_trap_original = 0 ds_no_memory = 0 ds_fragment_queue_fail = 0 ds_no_frag_entry = 0 ds_icmp_error = 0 ds_clone_fail = 0 ds_drop_pkts = 0 ] total_in_bandwidth_utilization = 4.87891e-17 total_out_bandwidth_utilization = 9.1073e-16 flow_rate= [ added_flows = 0 max_flow_adds_per_second = 0 min_flow_adds_per_second = 0 deleted_flows = 0 max_flow_deletes_per_second = 0 min_flow_deletes_per_second = 0 active_flows = 0 hold_flows = 0 ] phy_band_in_bps= [ [ _iter260->first = ens6 _iter260->second = 9, ] ] phy_band_out_bps= [ [ _iter261->first = ens6 _iter261->second = 168, ] ] raw_phy_if_stats= [ [ _iter262->first = ens6 [ in_pkts = 20 in_bytes = 1556 out_pkts = 112 out_bytes = 4844 ], ] ] raw_phy_if_drop_stats= [ [ _iter263->first = ens6 [ ds_discard = 0 ds_pull = 0 ds_invalid_if = 0 ds_invalid_arp = 0 ds_trap_no_if = 0 ds_nowhere_to_go = 0 ds_flow_queue_limit_exceeded = 0 ds_flow_no_memory = 0 ds_flow_invalid_protocol = 0 ds_flow_nat_no_rflow = 0 ds_flow_action_drop = 0 ds_flow_action_invalid = 0 ds_flow_unusable = 0 ds_flow_table_full = 0 ds_interface_tx_discard = 0 ds_interface_drop = 0 ds_duplicated = 0 ds_push = 0 ds_ttl_exceeded = 0 ds_invalid_nh = 0 ds_invalid_label = 0 ds_invalid_protocol = 0 ds_interface_rx_discard = 0 ds_invalid_mcast_source = 0 ds_head_alloc_fail = 0 ds_pcow_fail = 0 ds_mcast_clone_fail = 0 ds_rewrite_fail = 0 ds_misc = 0 ds_invalid_packet = 0 ds_cksum_err = 0 ds_no_fmd = 0 ds_invalid_vnid = 0 ds_frag_err = 0 ds_invalid_source = 0 ds_mcast_df_bit = 0 ds_l2_no_route = 0 ds_vlan_fwd_tx = 0 ds_vlan_fwd_enq = 0 ds_drop_new_flow = 0 ds_flow_evict = 0 ds_trap_original = 0 ds_no_memory = 0 ds_fragment_queue_fail = 0 ds_no_frag_entry = 0 ds_icmp_error = 0 ds_clone_fail = 0 ds_drop_pkts = 0 ], ] ] ] 2025-09-01 Mon 02:05:55:755.468 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848879547904, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: VrouterStats 2025-09-01 Mon 02:05:55:755.480 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848879547904, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:55:755.510 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848879547904, Pid 111888]: SANDESH: Send FAILED: 1756692355755307 [SYS_NOTICE]: VrouterControlStatsTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3992-1 raw_xmpp_stats= [ [ _iter306->first = 10.20.0.17 [ reconnects = 0 in_msgs = 0 out_msgs = 0 ], ] ] raw_ifmap_stats= [ [ _iter307->first = link_delete_parse_errors _iter307->second = 0, _iter307->first = link_update_parse_errors _iter307->second = 0, _iter307->first = node_delete_parse_errors _iter307->second = 0, _iter307->first = node_update_parse_errors _iter307->second = 0, ] ] raw_rt_table_size= [ [ _iter308->first = default-domain:default-project:ip-fabric:__default__ [ inet4_unicast = 4 inet4_multicast = 0 evpn = 0 bridge = 2 inet6_unicast = 2 ], _iter308->first = default-domain:default-project:ip-fabric:ip-fabric [ inet4_unicast = 2 inet4_multicast = 0 evpn = 1 bridge = 2 inet6_unicast = 0 ], ] ] ] 2025-09-01 Mon 02:05:55:755.526 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848879547904, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: VrouterControlStatsTrace 2025-09-01 Mon 02:05:56:762.251 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: Processing scm::EvIdleHoldTimerExpired in state Idle 2025-09-01 Mon 02:05:56:762.562 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: Connect : Start Connect timer 10.0.0.254:8086 2025-09-01 Mon 02:05:56:762.640 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848886536832, Pid 111888]: -(-1) OnSessionEvent TCP Connect Failed 2025-09-01 Mon 02:05:56:762.663 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: Processing scm::EvSandeshSend in state Connect 2025-09-01 Mon 02:05:56:762.724 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848886536832, Pid 111888]: -(-1):EvTcpConnectFail 2025-09-01 Mon 02:05:56:762.754 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: SANDESH: Send FAILED: 1756692356762337 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-3992-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.17 server_addrs= [ [ (*_iter6) = 10.20.0.17:5269, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.17 server_addrs= [ [ (*_iter6) = 10.20.0.17:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.254:8086, ] ] status = Initializing description = Idle : EvIdleHoldTimerExpired -> Connect ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.17, XMPP:dns-server:10.20.0.17, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:05:56:762.776 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: Wrong state: Connect for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:05:56:762.785 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: Processing scm::EvSandeshSend in state Connect 2025-09-01 Mon 02:05:56:762.940 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: SANDESH: Send FAILED: 1756692356762644 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3992-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Connect successful_connections = 0 pid = 111888 http_port = 8085 start_time = 1756692145743123 collector_name = collector_ip = 10.0.0.254:8086 collector_list= [ [ (*_iter40) = 10.0.0.254:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ComputeCpuStateTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 3 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 27 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 38 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 56 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterControlStatsTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 7 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 10 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 88 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 23 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 82, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 178, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:05:56:762.969 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: Wrong state: Connect for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:05:56:762.979 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: Processing scm::EvTcpConnectFail in state Connect 2025-09-01 Mon 02:05:56:762.985 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: Connect : EvTcpConnectFail 2025-09-01 Mon 02:05:56:763.042 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: TCP [SYS_ERR]: TcpSessionMessageLog: Session -(-1) > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:05:56:763.072 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: Idle 2025-09-01 Mon 02:05:56:763.133 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:56:763.160 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: SANDESH: Send FAILED: 1756692356763020 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-3992-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.17 server_addrs= [ [ (*_iter6) = 10.20.0.17:5269, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.17 server_addrs= [ [ (*_iter6) = 10.20.0.17:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.17, XMPP:dns-server:10.20.0.17, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:05:56:763.167 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:05:56:763.176 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: Processing scm::EvTcpDeleteSession in state Idle 2025-09-01 Mon 02:05:56:763.203 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:56:763.320 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: SANDESH: Send FAILED: 1756692356763112 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3992-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Idle successful_connections = 0 pid = 111888 http_port = 8085 start_time = 1756692145743123 collector_name = collector_ip = 10.0.0.254:8086 collector_list= [ [ (*_iter40) = 10.0.0.254:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ComputeCpuStateTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 3 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 28 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 39 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 57 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterControlStatsTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 7 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 10 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 88 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 23 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 83, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 180, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:05:56:763.336 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:05:59:921.134 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848879547904, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:59:921.263 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848879547904, Pid 111888]: SANDESH: Send FAILED: 1756692359921071 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.17 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:59:921.290 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848879547904, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:59:921.427 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848854357504, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:59:921.505 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848854357504, Pid 111888]: SANDESH: Send FAILED: 1756692359921374 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.17 state = Connect event = Connect Event local_port = 42759 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:59:921.535 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848886536832, Pid 111888]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.17 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:05:59:921.548 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848854357504, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:59:921.860 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848862754304, Pid 111888]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:05:59:922.004 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848858555904, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:59:922.044 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848858555904, Pid 111888]: SANDESH: Send FAILED: 1756692359921975 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.17 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:59:922.055 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848858555904, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:05:59:922.065 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848858555904, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:05:59:922.075 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848858555904, Pid 111888]: SANDESH: Send FAILED: 1756692359921996 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.17 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:05:59:922.082 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848858555904, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:08:096.512 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848886536832, Pid 111888]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.0.0.254:8085::10.0.0.254:45152 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 255 2025-09-01 Mon 02:06:15:233.178 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848862754304, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:15:233.354 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848862754304, Pid 111888]: SANDESH: Send FAILED: 1756692375233102 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.17 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:15:233.386 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848862754304, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:15:233.590 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:15:233.634 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848886536832, Pid 111888]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.17 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:06:15:233.843 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: SANDESH: Send FAILED: 1756692375233491 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.17 state = Connect event = Connect Event local_port = 37233 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:15:233.991 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:15:234.126 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848858555904, Pid 111888]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:06:15:234.351 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848879547904, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:15:234.466 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848879547904, Pid 111888]: SANDESH: Send FAILED: 1756692375234231 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.17 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:15:234.493 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848879547904, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:15:234.557 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848879547904, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:15:234.590 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848879547904, Pid 111888]: SANDESH: Send FAILED: 1756692375234330 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.17 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:15:234.609 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848879547904, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:21:111.138 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848886536832, Pid 111888]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.0.0.254:8085::10.0.0.254:51576 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 255 2025-09-01 Mon 02:06:25:750.146 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:25:750.340 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: SANDESH: Send FAILED: 1756692385749552 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3992-1:Compute:contrail-vrouter-agent:0 sm_queue_count = 0 sm_stats= [ ev_stats= [ [ [ event = scm::EvCollectorUpdate enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvIdleHoldTimerExpired enqueues = 11 dequeues = 11 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvSandeshSend enqueues = 190 dequeues = 190 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvStart enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpConnectFail enqueues = 12 dequeues = 12 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpDeleteSession enqueues = 12 dequeues = 12 enqueue_fails = 0 dequeue_fails = 0 ], [ event = enqueues = 227 dequeues = 227 enqueue_fails = 0 dequeue_fails = 0 ], ] ] state = Idle state_since = 1756692356763071 last_state = Idle last_event = scm::EvSandeshSend last_event_at = 1756692375234513 ] max_sm_queue_count = 6 ] 2025-09-01 Mon 02:06:25:750.379 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:06:25:750.405 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:25:750.873 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: SANDESH: Send FAILED: 1756692385750008 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3992-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Idle successful_connections = 0 pid = 111888 http_port = 8085 start_time = 1756692145743123 collector_name = collector_ip = 10.0.0.254:8086 collector_list= [ [ (*_iter40) = 10.0.0.254:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ComputeCpuStateTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 3 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 29 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 40 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 61 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterControlStatsTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 7 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 10 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 96 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 25 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 89, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 190, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:06:25:750.965 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:06:25:755.708 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:25:755.869 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: SANDESH: Send FAILED: 1756692385755638 [SYS_NOTICE]: VrouterStats: data= [ name = cn-jenkins-deploy-platform-ansible-os-3992-1 cpu_info= [ num_cpu = 8 sys_mem_info= [ total = 32 used = 0 free = 0 buffers = 0 cached = 510398136 node_type = ] meminfo= [ virt = 32561 peakvirt = 0 res = 0 ] cpuload= [ one_min_avg = 0.19875 five_min_avg = 0.23875 fifteen_min_avg = 0.2325 ] cpu_share = 0.466667 ] ] 2025-09-01 Mon 02:06:25:755.898 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: VrouterStats 2025-09-01 Mon 02:06:25:755.937 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:25:755.990 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: SANDESH: Send FAILED: 1756692385755666 [SYS_NOTICE]: ComputeCpuStateTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3992-1 cpu_info= [ [ [ mem_virt = 32561 cpu_share = 0.466667 used_sys_mem = 3784569160 one_min_cpuload = 0.19875 mem_res = 0 ], ] ] ] 2025-09-01 Mon 02:06:25:756.012 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: ComputeCpuStateTrace 2025-09-01 Mon 02:06:25:756.034 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:25:756.385 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: SANDESH: Send FAILED: 1756692385755713 [SYS_NOTICE]: VrouterStats: data= [ name = cn-jenkins-deploy-platform-ansible-os-3992-1 in_tpkts = 127 in_bytes = 5474 out_tpkts = 144 out_bytes = 6820 raw_vhost_stats= [ name = vhost0 in_pkts = 127 in_bytes = 5474 out_pkts = 144 out_bytes = 6820 speed = -1 duplexity = -1 ] raw_vhost_drop_stats= [ ds_discard = 0 ds_pull = 0 ds_invalid_if = 0 ds_invalid_arp = 0 ds_trap_no_if = 0 ds_nowhere_to_go = 0 ds_flow_queue_limit_exceeded = 0 ds_flow_no_memory = 0 ds_flow_invalid_protocol = 0 ds_flow_nat_no_rflow = 0 ds_flow_action_drop = 0 ds_flow_action_invalid = 0 ds_flow_unusable = 0 ds_flow_table_full = 0 ds_interface_tx_discard = 0 ds_interface_drop = 0 ds_duplicated = 0 ds_push = 0 ds_ttl_exceeded = 0 ds_invalid_nh = 0 ds_invalid_label = 0 ds_invalid_protocol = 0 ds_interface_rx_discard = 0 ds_invalid_mcast_source = 0 ds_head_alloc_fail = 0 ds_pcow_fail = 0 ds_mcast_clone_fail = 0 ds_rewrite_fail = 0 ds_misc = 0 ds_invalid_packet = 0 ds_cksum_err = 0 ds_no_fmd = 0 ds_invalid_vnid = 0 ds_frag_err = 0 ds_invalid_source = 0 ds_mcast_df_bit = 0 ds_l2_no_route = 0 ds_vlan_fwd_tx = 0 ds_vlan_fwd_enq = 0 ds_drop_new_flow = 0 ds_flow_evict = 0 ds_trap_original = 0 ds_no_memory = 0 ds_fragment_queue_fail = 0 ds_no_frag_entry = 0 ds_icmp_error = 0 ds_clone_fail = 0 ds_drop_pkts = 0 ] raw_drop_stats= [ ds_discard = 0 ds_pull = 0 ds_invalid_if = 0 ds_invalid_arp = 0 ds_trap_no_if = 0 ds_nowhere_to_go = 0 ds_flow_queue_limit_exceeded = 0 ds_flow_no_memory = 0 ds_flow_invalid_protocol = 0 ds_flow_nat_no_rflow = 0 ds_flow_action_drop = 0 ds_flow_action_invalid = 0 ds_flow_unusable = 0 ds_flow_table_full = 0 ds_interface_tx_discard = 0 ds_interface_drop = 0 ds_duplicated = 0 ds_push = 0 ds_ttl_exceeded = 0 ds_invalid_nh = 0 ds_invalid_label = 0 ds_invalid_protocol = 0 ds_interface_rx_discard = 0 ds_invalid_mcast_source = 0 ds_head_alloc_fail = 0 ds_pcow_fail = 0 ds_mcast_clone_fail = 0 ds_rewrite_fail = 0 ds_misc = 0 ds_invalid_packet = 0 ds_cksum_err = 0 ds_no_fmd = 0 ds_invalid_vnid = 0 ds_frag_err = 0 ds_invalid_source = 0 ds_mcast_df_bit = 0 ds_l2_no_route = 0 ds_vlan_fwd_tx = 0 ds_vlan_fwd_enq = 0 ds_drop_new_flow = 0 ds_flow_evict = 0 ds_trap_original = 0 ds_no_memory = 0 ds_fragment_queue_fail = 0 ds_no_frag_entry = 0 ds_icmp_error = 0 ds_clone_fail = 0 ds_drop_pkts = 0 ] flow_rate= [ added_flows = 0 max_flow_adds_per_second = 0 min_flow_adds_per_second = 0 deleted_flows = 0 max_flow_deletes_per_second = 0 min_flow_deletes_per_second = 0 active_flows = 0 hold_flows = 0 ] raw_phy_if_stats= [ [ _iter262->first = ens6 [ in_pkts = 22 in_bytes = 1696 out_pkts = 127 out_bytes = 5474 ], ] ] raw_phy_if_drop_stats= [ [ _iter263->first = ens6 [ ds_discard = 0 ds_pull = 0 ds_invalid_if = 0 ds_invalid_arp = 0 ds_trap_no_if = 0 ds_nowhere_to_go = 0 ds_flow_queue_limit_exceeded = 0 ds_flow_no_memory = 0 ds_flow_invalid_protocol = 0 ds_flow_nat_no_rflow = 0 ds_flow_action_drop = 0 ds_flow_action_invalid = 0 ds_flow_unusable = 0 ds_flow_table_full = 0 ds_interface_tx_discard = 0 ds_interface_drop = 0 ds_duplicated = 0 ds_push = 0 ds_ttl_exceeded = 0 ds_invalid_nh = 0 ds_invalid_label = 0 ds_invalid_protocol = 0 ds_interface_rx_discard = 0 ds_invalid_mcast_source = 0 ds_head_alloc_fail = 0 ds_pcow_fail = 0 ds_mcast_clone_fail = 0 ds_rewrite_fail = 0 ds_misc = 0 ds_invalid_packet = 0 ds_cksum_err = 0 ds_no_fmd = 0 ds_invalid_vnid = 0 ds_frag_err = 0 ds_invalid_source = 0 ds_mcast_df_bit = 0 ds_l2_no_route = 0 ds_vlan_fwd_tx = 0 ds_vlan_fwd_enq = 0 ds_drop_new_flow = 0 ds_flow_evict = 0 ds_trap_original = 0 ds_no_memory = 0 ds_fragment_queue_fail = 0 ds_no_frag_entry = 0 ds_icmp_error = 0 ds_clone_fail = 0 ds_drop_pkts = 0 ], ] ] ] 2025-09-01 Mon 02:06:25:756.445 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: VrouterStats 2025-09-01 Mon 02:06:25:756.474 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:25:756.571 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: SANDESH: Send FAILED: 1756692385755764 [SYS_NOTICE]: VrouterControlStatsTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3992-1 raw_xmpp_stats= [ [ _iter306->first = 10.20.0.17 [ reconnects = 0 in_msgs = 0 out_msgs = 0 ], ] ] raw_ifmap_stats= [ [ _iter307->first = link_delete_parse_errors _iter307->second = 0, _iter307->first = link_update_parse_errors _iter307->second = 0, _iter307->first = node_delete_parse_errors _iter307->second = 0, _iter307->first = node_update_parse_errors _iter307->second = 0, ] ] raw_rt_table_size= [ [ _iter308->first = default-domain:default-project:ip-fabric:__default__ [ inet4_unicast = 4 inet4_multicast = 0 evpn = 0 bridge = 2 inet6_unicast = 2 ], _iter308->first = default-domain:default-project:ip-fabric:ip-fabric [ inet4_unicast = 2 inet4_multicast = 0 evpn = 1 bridge = 2 inet6_unicast = 0 ], ] ] ] 2025-09-01 Mon 02:06:25:756.593 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: VrouterControlStatsTrace 2025-09-01 Mon 02:06:26:763.340 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848854357504, Pid 111888]: Processing scm::EvIdleHoldTimerExpired in state Idle 2025-09-01 Mon 02:06:26:763.761 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848854357504, Pid 111888]: Connect : Start Connect timer 10.0.0.254:8086 2025-09-01 Mon 02:06:26:763.838 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848886536832, Pid 111888]: -(-1) OnSessionEvent TCP Connect Failed 2025-09-01 Mon 02:06:26:763.921 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848886536832, Pid 111888]: -(-1):EvTcpConnectFail 2025-09-01 Mon 02:06:26:763.940 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848854357504, Pid 111888]: Processing scm::EvSandeshSend in state Connect 2025-09-01 Mon 02:06:26:763.999 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848854357504, Pid 111888]: SANDESH: Send FAILED: 1756692386763480 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-3992-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.17 server_addrs= [ [ (*_iter6) = 10.20.0.17:5269, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.17 server_addrs= [ [ (*_iter6) = 10.20.0.17:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.254:8086, ] ] status = Initializing description = Idle : EvIdleHoldTimerExpired -> Connect ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.17, XMPP:dns-server:10.20.0.17, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:06:26:764.013 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848854357504, Pid 111888]: Wrong state: Connect for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:06:26:764.027 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848854357504, Pid 111888]: Processing scm::EvSandeshSend in state Connect 2025-09-01 Mon 02:06:26:764.300 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848854357504, Pid 111888]: SANDESH: Send FAILED: 1756692386763905 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3992-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Connect successful_connections = 0 pid = 111888 http_port = 8085 start_time = 1756692145743123 collector_name = collector_ip = 10.0.0.254:8086 collector_list= [ [ (*_iter40) = 10.0.0.254:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ComputeCpuStateTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 4 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 29 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 42 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 61 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterControlStatsTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 8 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 12 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 96 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 25 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 89, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 196, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:06:26:764.331 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848854357504, Pid 111888]: Wrong state: Connect for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:06:26:764.347 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848854357504, Pid 111888]: Processing scm::EvTcpConnectFail in state Connect 2025-09-01 Mon 02:06:26:764.358 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848854357504, Pid 111888]: Connect : EvTcpConnectFail 2025-09-01 Mon 02:06:26:764.434 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848854357504, Pid 111888]: TCP [SYS_ERR]: TcpSessionMessageLog: Session -(-1) > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:06:26:764.486 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848854357504, Pid 111888]: Idle 2025-09-01 Mon 02:06:26:764.591 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848854357504, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:26:764.623 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848854357504, Pid 111888]: SANDESH: Send FAILED: 1756692386764391 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-3992-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.17 server_addrs= [ [ (*_iter6) = 10.20.0.17:5269, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.17 server_addrs= [ [ (*_iter6) = 10.20.0.17:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.17, XMPP:dns-server:10.20.0.17, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:06:26:764.641 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848854357504, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:06:26:764.653 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848854357504, Pid 111888]: Processing scm::EvTcpDeleteSession in state Idle 2025-09-01 Mon 02:06:26:764.691 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848854357504, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:26:764.958 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848854357504, Pid 111888]: SANDESH: Send FAILED: 1756692386764573 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3992-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Idle successful_connections = 0 pid = 111888 http_port = 8085 start_time = 1756692145743123 collector_name = collector_ip = 10.0.0.254:8086 collector_list= [ [ (*_iter40) = 10.0.0.254:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ComputeCpuStateTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 4 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 30 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 43 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 62 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterControlStatsTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 8 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 12 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 96 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 25 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 90, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 198, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:06:26:764.993 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848854357504, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:06:27:667.761 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848886536832, Pid 111888]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.20.0.17:8085::10.20.0.17:52984 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 255 2025-09-01 Mon 02:06:27:686.815 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848886536832, Pid 111888]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.20.0.17:8085::10.20.0.17:53000 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 255 2025-09-01 Mon 02:06:28:542.503 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:28:542.653 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: SANDESH: Send FAILED: 1756692388542332 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.17 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:28:542.681 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:28:542.785 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848886536832, Pid 111888]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.17 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:06:28:542.830 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848854357504, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:28:542.887 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:06:28:542.955 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848854357504, Pid 111888]: SANDESH: Send FAILED: 1756692388542746 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.17 state = Connect event = Connect Event local_port = 33441 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:28:542.986 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848854357504, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:28:543.047 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848854357504, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:28:543.079 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848854357504, Pid 111888]: SANDESH: Send FAILED: 1756692388542963 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.17 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:28:543.098 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848854357504, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:28:543.118 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848854357504, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:28:543.146 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848854357504, Pid 111888]: SANDESH: Send FAILED: 1756692388542969 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.17 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:28:543.164 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848854357504, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:33:878.024 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848886536832, Pid 111888]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.0.0.254:8085::10.0.0.254:54788 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 255 2025-09-01 Mon 02:06:42:774.800 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848879547904, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:42:774.930 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848879547904, Pid 111888]: SANDESH: Send FAILED: 1756692402774709 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.17 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:42:774.959 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848879547904, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:42:775.261 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848862754304, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:42:775.394 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848862754304, Pid 111888]: SANDESH: Send FAILED: 1756692402775177 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.17 state = Connect event = Connect Event local_port = 44289 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:42:775.424 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848862754304, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:42:775.264 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848886536832, Pid 111888]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.17 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:06:42:775.659 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848858555904, Pid 111888]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:06:42:775.955 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848854357504, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:42:776.087 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848854357504, Pid 111888]: SANDESH: Send FAILED: 1756692402775793 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.17 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:42:776.147 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848854357504, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:42:776.175 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848854357504, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:42:776.205 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848854357504, Pid 111888]: SANDESH: Send FAILED: 1756692402775845 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.17 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:42:776.223 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848854357504, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:46:781.244 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848886536832, Pid 111888]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.0.0.254:8085::10.0.0.254:33928 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 255 2025-09-01 Mon 02:06:55:750.651 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848866952704, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:55:750.943 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848866952704, Pid 111888]: SANDESH: Send FAILED: 1756692415750259 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3992-1:Compute:contrail-vrouter-agent:0 sm_queue_count = 0 sm_stats= [ ev_stats= [ [ [ event = scm::EvCollectorUpdate enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvIdleHoldTimerExpired enqueues = 12 dequeues = 12 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvSandeshSend enqueues = 208 dequeues = 208 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvStart enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpConnectFail enqueues = 13 dequeues = 13 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpDeleteSession enqueues = 13 dequeues = 13 enqueue_fails = 0 dequeue_fails = 0 ], [ event = enqueues = 248 dequeues = 248 enqueue_fails = 0 dequeue_fails = 0 ], ] ] state = Idle state_since = 1756692386764485 last_state = Idle last_event = scm::EvSandeshSend last_event_at = 1756692402776171 ] max_sm_queue_count = 6 ] 2025-09-01 Mon 02:06:55:750.994 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848866952704, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:06:55:751.031 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848866952704, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:55:751.701 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848866952704, Pid 111888]: SANDESH: Send FAILED: 1756692415750510 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3992-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Idle successful_connections = 0 pid = 111888 http_port = 8085 start_time = 1756692145743123 collector_name = collector_ip = 10.0.0.254:8086 collector_list= [ [ (*_iter40) = 10.0.0.254:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ComputeCpuStateTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 4 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 31 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 44 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 68 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterControlStatsTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 8 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 12 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 104 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 27 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 98, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 208, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:06:55:751.772 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848866952704, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:06:55:756.232 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848879547904, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:55:756.711 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848879547904, Pid 111888]: SANDESH: Send FAILED: 1756692415756118 [SYS_NOTICE]: VrouterStats: data= [ name = cn-jenkins-deploy-platform-ansible-os-3992-1 in_tpkts = 143 in_bytes = 6174 out_tpkts = 159 out_bytes = 7450 exception_packets = 12 exception_packets_dropped = 12 raw_vhost_stats= [ name = vhost0 in_pkts = 143 in_bytes = 6174 out_pkts = 159 out_bytes = 7450 speed = -1 duplexity = -1 ] raw_vhost_drop_stats= [ ds_discard = 0 ds_pull = 0 ds_invalid_if = 0 ds_invalid_arp = 0 ds_trap_no_if = 0 ds_nowhere_to_go = 0 ds_flow_queue_limit_exceeded = 0 ds_flow_no_memory = 0 ds_flow_invalid_protocol = 0 ds_flow_nat_no_rflow = 0 ds_flow_action_drop = 0 ds_flow_action_invalid = 0 ds_flow_unusable = 0 ds_flow_table_full = 0 ds_interface_tx_discard = 0 ds_interface_drop = 0 ds_duplicated = 0 ds_push = 0 ds_ttl_exceeded = 0 ds_invalid_nh = 0 ds_invalid_label = 0 ds_invalid_protocol = 0 ds_interface_rx_discard = 0 ds_invalid_mcast_source = 0 ds_head_alloc_fail = 0 ds_pcow_fail = 0 ds_mcast_clone_fail = 0 ds_rewrite_fail = 0 ds_misc = 0 ds_invalid_packet = 0 ds_cksum_err = 0 ds_no_fmd = 0 ds_invalid_vnid = 0 ds_frag_err = 0 ds_invalid_source = 0 ds_mcast_df_bit = 0 ds_l2_no_route = 0 ds_vlan_fwd_tx = 0 ds_vlan_fwd_enq = 0 ds_drop_new_flow = 0 ds_flow_evict = 0 ds_trap_original = 0 ds_no_memory = 0 ds_fragment_queue_fail = 0 ds_no_frag_entry = 0 ds_icmp_error = 0 ds_clone_fail = 0 ds_drop_pkts = 0 ] raw_drop_stats= [ ds_discard = 0 ds_pull = 0 ds_invalid_if = 0 ds_invalid_arp = 0 ds_trap_no_if = 0 ds_nowhere_to_go = 0 ds_flow_queue_limit_exceeded = 0 ds_flow_no_memory = 0 ds_flow_invalid_protocol = 0 ds_flow_nat_no_rflow = 0 ds_flow_action_drop = 0 ds_flow_action_invalid = 0 ds_flow_unusable = 0 ds_flow_table_full = 0 ds_interface_tx_discard = 0 ds_interface_drop = 0 ds_duplicated = 0 ds_push = 0 ds_ttl_exceeded = 0 ds_invalid_nh = 0 ds_invalid_label = 0 ds_invalid_protocol = 0 ds_interface_rx_discard = 0 ds_invalid_mcast_source = 0 ds_head_alloc_fail = 0 ds_pcow_fail = 0 ds_mcast_clone_fail = 0 ds_rewrite_fail = 0 ds_misc = 0 ds_invalid_packet = 0 ds_cksum_err = 0 ds_no_fmd = 0 ds_invalid_vnid = 0 ds_frag_err = 0 ds_invalid_source = 0 ds_mcast_df_bit = 0 ds_l2_no_route = 0 ds_vlan_fwd_tx = 0 ds_vlan_fwd_enq = 0 ds_drop_new_flow = 0 ds_flow_evict = 0 ds_trap_original = 0 ds_no_memory = 0 ds_fragment_queue_fail = 0 ds_no_frag_entry = 0 ds_icmp_error = 0 ds_clone_fail = 0 ds_drop_pkts = 0 ] total_in_bandwidth_utilization = 9.75782e-17 total_out_bandwidth_utilization = 9.59519e-16 flow_rate= [ added_flows = 0 max_flow_adds_per_second = 0 min_flow_adds_per_second = 0 deleted_flows = 0 max_flow_deletes_per_second = 0 min_flow_deletes_per_second = 0 active_flows = 0 hold_flows = 0 ] phy_band_in_bps= [ [ _iter260->first = ens6 _iter260->second = 18, ] ] phy_band_out_bps= [ [ _iter261->first = ens6 _iter261->second = 177, ] ] raw_phy_if_stats= [ [ _iter262->first = ens6 [ in_pkts = 22 in_bytes = 1696 out_pkts = 143 out_bytes = 6174 ], ] ] raw_phy_if_drop_stats= [ [ _iter263->first = ens6 [ ds_discard = 0 ds_pull = 0 ds_invalid_if = 0 ds_invalid_arp = 0 ds_trap_no_if = 0 ds_nowhere_to_go = 0 ds_flow_queue_limit_exceeded = 0 ds_flow_no_memory = 0 ds_flow_invalid_protocol = 0 ds_flow_nat_no_rflow = 0 ds_flow_action_drop = 0 ds_flow_action_invalid = 0 ds_flow_unusable = 0 ds_flow_table_full = 0 ds_interface_tx_discard = 0 ds_interface_drop = 0 ds_duplicated = 0 ds_push = 0 ds_ttl_exceeded = 0 ds_invalid_nh = 0 ds_invalid_label = 0 ds_invalid_protocol = 0 ds_interface_rx_discard = 0 ds_invalid_mcast_source = 0 ds_head_alloc_fail = 0 ds_pcow_fail = 0 ds_mcast_clone_fail = 0 ds_rewrite_fail = 0 ds_misc = 0 ds_invalid_packet = 0 ds_cksum_err = 0 ds_no_fmd = 0 ds_invalid_vnid = 0 ds_frag_err = 0 ds_invalid_source = 0 ds_mcast_df_bit = 0 ds_l2_no_route = 0 ds_vlan_fwd_tx = 0 ds_vlan_fwd_enq = 0 ds_drop_new_flow = 0 ds_flow_evict = 0 ds_trap_original = 0 ds_no_memory = 0 ds_fragment_queue_fail = 0 ds_no_frag_entry = 0 ds_icmp_error = 0 ds_clone_fail = 0 ds_drop_pkts = 0 ], ] ] ] 2025-09-01 Mon 02:06:55:756.765 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848879547904, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: VrouterStats 2025-09-01 Mon 02:06:55:756.801 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848879547904, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:55:756.874 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848879547904, Pid 111888]: SANDESH: Send FAILED: 1756692415756279 [SYS_NOTICE]: VrouterControlStatsTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3992-1 raw_xmpp_stats= [ [ _iter306->first = 10.20.0.17 [ reconnects = 0 in_msgs = 0 out_msgs = 0 ], ] ] raw_ifmap_stats= [ [ _iter307->first = link_delete_parse_errors _iter307->second = 0, _iter307->first = link_update_parse_errors _iter307->second = 0, _iter307->first = node_delete_parse_errors _iter307->second = 0, _iter307->first = node_update_parse_errors _iter307->second = 0, ] ] raw_rt_table_size= [ [ _iter308->first = default-domain:default-project:ip-fabric:__default__ [ inet4_unicast = 4 inet4_multicast = 0 evpn = 0 bridge = 2 inet6_unicast = 2 ], _iter308->first = default-domain:default-project:ip-fabric:ip-fabric [ inet4_unicast = 2 inet4_multicast = 0 evpn = 1 bridge = 2 inet6_unicast = 0 ], ] ] ] 2025-09-01 Mon 02:06:55:756.894 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848879547904, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: VrouterControlStatsTrace 2025-09-01 Mon 02:06:56:764.784 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: Processing scm::EvIdleHoldTimerExpired in state Idle 2025-09-01 Mon 02:06:56:765.385 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: Connect : Start Connect timer 10.0.0.254:8086 2025-09-01 Mon 02:06:56:765.391 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848886536832, Pid 111888]: -(-1) OnSessionEvent TCP Connect Failed 2025-09-01 Mon 02:06:56:765.471 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848886536832, Pid 111888]: -(-1):EvTcpConnectFail 2025-09-01 Mon 02:06:56:765.749 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: Processing scm::EvSandeshSend in state Connect 2025-09-01 Mon 02:06:56:765.852 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: SANDESH: Send FAILED: 1756692416765044 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-3992-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.17 server_addrs= [ [ (*_iter6) = 10.20.0.17:5269, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.17 server_addrs= [ [ (*_iter6) = 10.20.0.17:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.254:8086, ] ] status = Initializing description = Idle : EvIdleHoldTimerExpired -> Connect ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.17, XMPP:dns-server:10.20.0.17, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:06:56:765.880 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: Wrong state: Connect for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:06:56:765.907 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: Processing scm::EvTcpConnectFail in state Connect 2025-09-01 Mon 02:06:56:765.928 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: Connect : EvTcpConnectFail 2025-09-01 Mon 02:06:56:766.037 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: TCP [SYS_ERR]: TcpSessionMessageLog: Session -(-1) > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:06:56:766.127 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: Idle 2025-09-01 Mon 02:06:56:766.273 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:56:766.726 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: SANDESH: Send FAILED: 1756692416765635 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3992-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Connect successful_connections = 0 pid = 111888 http_port = 8085 start_time = 1756692145743123 collector_name = collector_ip = 10.0.0.254:8086 collector_list= [ [ (*_iter40) = 10.0.0.254:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ComputeCpuStateTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 4 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 31 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 46 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 68 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterControlStatsTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 9 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 13 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 104 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 27 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 98, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 212, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:06:56:767.372 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:06:56:767.410 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:56:767.464 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: SANDESH: Send FAILED: 1756692416765984 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-3992-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.17 server_addrs= [ [ (*_iter6) = 10.20.0.17:5269, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.17 server_addrs= [ [ (*_iter6) = 10.20.0.17:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.17, XMPP:dns-server:10.20.0.17, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:06:56:767.483 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:06:56:767.506 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: Processing scm::EvTcpDeleteSession in state Idle 2025-09-01 Mon 02:06:56:767.600 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:56:767.983 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: SANDESH: Send FAILED: 1756692416766238 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3992-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Idle successful_connections = 0 pid = 111888 http_port = 8085 start_time = 1756692145743123 collector_name = collector_ip = 10.0.0.254:8086 collector_list= [ [ (*_iter40) = 10.0.0.254:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ComputeCpuStateTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 4 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 32 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 46 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 69 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterControlStatsTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 9 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 13 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 104 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 27 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 99, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 213, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:06:56:768.032 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:06:58:783.239 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848862754304, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:58:783.320 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848862754304, Pid 111888]: SANDESH: Send FAILED: 1756692418783200 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.17 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:58:783.350 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848862754304, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:58:783.471 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848886536832, Pid 111888]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.17 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:06:58:783.639 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848858555904, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:58:783.680 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848858555904, Pid 111888]: SANDESH: Send FAILED: 1756692418783364 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.17 state = Connect event = Connect Event local_port = 41105 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:58:783.691 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848858555904, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:58:783.701 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848866952704, Pid 111888]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:06:58:783.783 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848858555904, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:58:783.799 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848858555904, Pid 111888]: SANDESH: Send FAILED: 1756692418783770 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.17 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:58:783.815 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848858555904, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:58:783.824 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848858555904, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:06:58:783.836 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848858555904, Pid 111888]: SANDESH: Send FAILED: 1756692418783791 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.17 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:06:58:783.856 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848858555904, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:06:59:971.951 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848886536832, Pid 111888]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.0.0.254:8085::10.0.0.254:40152 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 255 2025-09-01 Mon 02:07:12:959.641 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848886536832, Pid 111888]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.0.0.254:8085::10.0.0.254:42436 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 255 2025-09-01 Mon 02:07:14:366.201 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848854357504, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:14:366.277 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848854357504, Pid 111888]: SANDESH: Send FAILED: 1756692434366142 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.17 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:14:366.302 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848854357504, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:14:366.499 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:14:366.617 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: SANDESH: Send FAILED: 1756692434366422 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.17 state = Connect event = Connect Event local_port = 43213 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:14:366.648 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:14:366.516 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848886536832, Pid 111888]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.17 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:07:14:366.894 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:07:14:367.012 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848862754304, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:14:367.106 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848862754304, Pid 111888]: SANDESH: Send FAILED: 1756692434366948 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.17 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:14:367.132 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848862754304, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:14:367.158 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848862754304, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:14:367.189 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848862754304, Pid 111888]: SANDESH: Send FAILED: 1756692434366958 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.17 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:14:367.216 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848862754304, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:25:750.922 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848866952704, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:25:751.005 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848866952704, Pid 111888]: SANDESH: Send FAILED: 1756692445750771 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3992-1:Compute:contrail-vrouter-agent:0 sm_queue_count = 0 sm_stats= [ ev_stats= [ [ [ event = scm::EvCollectorUpdate enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvIdleHoldTimerExpired enqueues = 13 dequeues = 13 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvSandeshSend enqueues = 224 dequeues = 224 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvStart enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpConnectFail enqueues = 14 dequeues = 14 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpDeleteSession enqueues = 14 dequeues = 14 enqueue_fails = 0 dequeue_fails = 0 ], [ event = enqueues = 267 dequeues = 267 enqueue_fails = 0 dequeue_fails = 0 ], ] ] state = Idle state_since = 1756692416766124 last_state = Idle last_event = scm::EvSandeshSend last_event_at = 1756692434367153 ] max_sm_queue_count = 6 ] 2025-09-01 Mon 02:07:25:751.030 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848866952704, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:07:25:751.040 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848866952704, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:25:751.205 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848866952704, Pid 111888]: SANDESH: Send FAILED: 1756692445750872 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3992-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Idle successful_connections = 0 pid = 111888 http_port = 8085 start_time = 1756692145743123 collector_name = collector_ip = 10.0.0.254:8086 collector_list= [ [ (*_iter40) = 10.0.0.254:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ComputeCpuStateTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 4 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 33 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 48 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 73 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterControlStatsTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 9 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 13 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 112 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 29 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 105, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 224, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:07:25:751.226 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848866952704, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:07:25:756.465 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848854357504, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:25:756.575 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848854357504, Pid 111888]: SANDESH: Send FAILED: 1756692445756426 [SYS_NOTICE]: VrouterStats: data= [ name = cn-jenkins-deploy-platform-ansible-os-3992-1 cpu_info= [ num_cpu = 8 sys_mem_info= [ total = 32 used = 0 free = 0 buffers = 0 cached = 510398136 node_type = ] meminfo= [ virt = 32561 peakvirt = 0 res = 0 ] cpuload= [ one_min_avg = 0.2375 five_min_avg = 0.23875 fifteen_min_avg = 0.2325 ] cpu_share = 0.452083 ] ] 2025-09-01 Mon 02:07:25:756.593 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848854357504, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: VrouterStats 2025-09-01 Mon 02:07:25:756.610 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848854357504, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:25:756.637 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848854357504, Pid 111888]: SANDESH: Send FAILED: 1756692445756443 [SYS_NOTICE]: ComputeCpuStateTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3992-1 cpu_info= [ [ [ mem_virt = 32561 cpu_share = 0.452083 used_sys_mem = 3784569160 one_min_cpuload = 0.2375 mem_res = 0 ], ] ] ] 2025-09-01 Mon 02:07:25:756.649 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848854357504, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: ComputeCpuStateTrace 2025-09-01 Mon 02:07:25:756.662 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848854357504, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:25:756.848 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848854357504, Pid 111888]: SANDESH: Send FAILED: 1756692445756467 [SYS_NOTICE]: VrouterStats: data= [ name = cn-jenkins-deploy-platform-ansible-os-3992-1 in_tpkts = 156 in_bytes = 6720 out_tpkts = 173 out_bytes = 8066 raw_vhost_stats= [ name = vhost0 in_pkts = 156 in_bytes = 6720 out_pkts = 173 out_bytes = 8066 speed = -1 duplexity = -1 ] raw_vhost_drop_stats= [ ds_discard = 0 ds_pull = 0 ds_invalid_if = 0 ds_invalid_arp = 0 ds_trap_no_if = 0 ds_nowhere_to_go = 0 ds_flow_queue_limit_exceeded = 0 ds_flow_no_memory = 0 ds_flow_invalid_protocol = 0 ds_flow_nat_no_rflow = 0 ds_flow_action_drop = 0 ds_flow_action_invalid = 0 ds_flow_unusable = 0 ds_flow_table_full = 0 ds_interface_tx_discard = 0 ds_interface_drop = 0 ds_duplicated = 0 ds_push = 0 ds_ttl_exceeded = 0 ds_invalid_nh = 0 ds_invalid_label = 0 ds_invalid_protocol = 0 ds_interface_rx_discard = 0 ds_invalid_mcast_source = 0 ds_head_alloc_fail = 0 ds_pcow_fail = 0 ds_mcast_clone_fail = 0 ds_rewrite_fail = 0 ds_misc = 0 ds_invalid_packet = 0 ds_cksum_err = 0 ds_no_fmd = 0 ds_invalid_vnid = 0 ds_frag_err = 0 ds_invalid_source = 0 ds_mcast_df_bit = 0 ds_l2_no_route = 0 ds_vlan_fwd_tx = 0 ds_vlan_fwd_enq = 0 ds_drop_new_flow = 0 ds_flow_evict = 0 ds_trap_original = 0 ds_no_memory = 0 ds_fragment_queue_fail = 0 ds_no_frag_entry = 0 ds_icmp_error = 0 ds_clone_fail = 0 ds_drop_pkts = 0 ] raw_drop_stats= [ ds_discard = 0 ds_pull = 0 ds_invalid_if = 0 ds_invalid_arp = 0 ds_trap_no_if = 0 ds_nowhere_to_go = 0 ds_flow_queue_limit_exceeded = 0 ds_flow_no_memory = 0 ds_flow_invalid_protocol = 0 ds_flow_nat_no_rflow = 0 ds_flow_action_drop = 0 ds_flow_action_invalid = 0 ds_flow_unusable = 0 ds_flow_table_full = 0 ds_interface_tx_discard = 0 ds_interface_drop = 0 ds_duplicated = 0 ds_push = 0 ds_ttl_exceeded = 0 ds_invalid_nh = 0 ds_invalid_label = 0 ds_invalid_protocol = 0 ds_interface_rx_discard = 0 ds_invalid_mcast_source = 0 ds_head_alloc_fail = 0 ds_pcow_fail = 0 ds_mcast_clone_fail = 0 ds_rewrite_fail = 0 ds_misc = 0 ds_invalid_packet = 0 ds_cksum_err = 0 ds_no_fmd = 0 ds_invalid_vnid = 0 ds_frag_err = 0 ds_invalid_source = 0 ds_mcast_df_bit = 0 ds_l2_no_route = 0 ds_vlan_fwd_tx = 0 ds_vlan_fwd_enq = 0 ds_drop_new_flow = 0 ds_flow_evict = 0 ds_trap_original = 0 ds_no_memory = 0 ds_fragment_queue_fail = 0 ds_no_frag_entry = 0 ds_icmp_error = 0 ds_clone_fail = 0 ds_drop_pkts = 0 ] flow_rate= [ added_flows = 0 max_flow_adds_per_second = 0 min_flow_adds_per_second = 0 deleted_flows = 0 max_flow_deletes_per_second = 0 min_flow_deletes_per_second = 0 active_flows = 0 hold_flows = 0 ] raw_phy_if_stats= [ [ _iter262->first = ens6 [ in_pkts = 23 in_bytes = 1766 out_pkts = 156 out_bytes = 6720 ], ] ] raw_phy_if_drop_stats= [ [ _iter263->first = ens6 [ ds_discard = 0 ds_pull = 0 ds_invalid_if = 0 ds_invalid_arp = 0 ds_trap_no_if = 0 ds_nowhere_to_go = 0 ds_flow_queue_limit_exceeded = 0 ds_flow_no_memory = 0 ds_flow_invalid_protocol = 0 ds_flow_nat_no_rflow = 0 ds_flow_action_drop = 0 ds_flow_action_invalid = 0 ds_flow_unusable = 0 ds_flow_table_full = 0 ds_interface_tx_discard = 0 ds_interface_drop = 0 ds_duplicated = 0 ds_push = 0 ds_ttl_exceeded = 0 ds_invalid_nh = 0 ds_invalid_label = 0 ds_invalid_protocol = 0 ds_interface_rx_discard = 0 ds_invalid_mcast_source = 0 ds_head_alloc_fail = 0 ds_pcow_fail = 0 ds_mcast_clone_fail = 0 ds_rewrite_fail = 0 ds_misc = 0 ds_invalid_packet = 0 ds_cksum_err = 0 ds_no_fmd = 0 ds_invalid_vnid = 0 ds_frag_err = 0 ds_invalid_source = 0 ds_mcast_df_bit = 0 ds_l2_no_route = 0 ds_vlan_fwd_tx = 0 ds_vlan_fwd_enq = 0 ds_drop_new_flow = 0 ds_flow_evict = 0 ds_trap_original = 0 ds_no_memory = 0 ds_fragment_queue_fail = 0 ds_no_frag_entry = 0 ds_icmp_error = 0 ds_clone_fail = 0 ds_drop_pkts = 0 ], ] ] ] 2025-09-01 Mon 02:07:25:756.881 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848854357504, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: VrouterStats 2025-09-01 Mon 02:07:25:756.895 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848854357504, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:25:756.946 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848854357504, Pid 111888]: SANDESH: Send FAILED: 1756692445756519 [SYS_NOTICE]: VrouterControlStatsTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3992-1 raw_xmpp_stats= [ [ _iter306->first = 10.20.0.17 [ reconnects = 0 in_msgs = 0 out_msgs = 0 ], ] ] raw_ifmap_stats= [ [ _iter307->first = link_delete_parse_errors _iter307->second = 0, _iter307->first = link_update_parse_errors _iter307->second = 0, _iter307->first = node_delete_parse_errors _iter307->second = 0, _iter307->first = node_update_parse_errors _iter307->second = 0, ] ] raw_rt_table_size= [ [ _iter308->first = default-domain:default-project:ip-fabric:__default__ [ inet4_unicast = 4 inet4_multicast = 0 evpn = 0 bridge = 2 inet6_unicast = 2 ], _iter308->first = default-domain:default-project:ip-fabric:ip-fabric [ inet4_unicast = 2 inet4_multicast = 0 evpn = 1 bridge = 2 inet6_unicast = 0 ], ] ] ] 2025-09-01 Mon 02:07:25:756.972 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848854357504, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: VrouterControlStatsTrace 2025-09-01 Mon 02:07:26:081.270 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848886536832, Pid 111888]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.0.0.254:8085::10.0.0.254:48562 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 255 2025-09-01 Mon 02:07:26:766.371 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848879547904, Pid 111888]: Processing scm::EvIdleHoldTimerExpired in state Idle 2025-09-01 Mon 02:07:26:766.879 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848879547904, Pid 111888]: Connect : Start Connect timer 10.0.0.254:8086 2025-09-01 Mon 02:07:26:766.935 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848886536832, Pid 111888]: -(-1) OnSessionEvent TCP Connect Failed 2025-09-01 Mon 02:07:26:767.009 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848886536832, Pid 111888]: -(-1):EvTcpConnectFail 2025-09-01 Mon 02:07:26:767.112 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848879547904, Pid 111888]: Processing scm::EvSandeshSend in state Connect 2025-09-01 Mon 02:07:26:767.175 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848879547904, Pid 111888]: SANDESH: Send FAILED: 1756692446766509 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-3992-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.17 server_addrs= [ [ (*_iter6) = 10.20.0.17:5269, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.17 server_addrs= [ [ (*_iter6) = 10.20.0.17:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.254:8086, ] ] status = Initializing description = Idle : EvIdleHoldTimerExpired -> Connect ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.17, XMPP:dns-server:10.20.0.17, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:07:26:767.198 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848879547904, Pid 111888]: Wrong state: Connect for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:07:26:767.222 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848879547904, Pid 111888]: Processing scm::EvTcpConnectFail in state Connect 2025-09-01 Mon 02:07:26:767.242 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848879547904, Pid 111888]: Connect : EvTcpConnectFail 2025-09-01 Mon 02:07:26:767.333 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848879547904, Pid 111888]: TCP [SYS_ERR]: TcpSessionMessageLog: Session -(-1) > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:07:26:767.396 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848879547904, Pid 111888]: Idle 2025-09-01 Mon 02:07:26:767.587 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848879547904, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:26:767.979 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848879547904, Pid 111888]: SANDESH: Send FAILED: 1756692446767076 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3992-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Connect successful_connections = 0 pid = 111888 http_port = 8085 start_time = 1756692145743123 collector_name = collector_ip = 10.0.0.254:8086 collector_list= [ [ (*_iter40) = 10.0.0.254:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ComputeCpuStateTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 5 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 33 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 50 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 74 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterControlStatsTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 10 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 15 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 112 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 29 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 106, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 230, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:07:26:768.044 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848879547904, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:07:26:768.073 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848879547904, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:26:768.120 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848879547904, Pid 111888]: SANDESH: Send FAILED: 1756692446767289 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-3992-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.17 server_addrs= [ [ (*_iter6) = 10.20.0.17:5269, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.17 server_addrs= [ [ (*_iter6) = 10.20.0.17:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.17, XMPP:dns-server:10.20.0.17, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:07:26:768.158 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848879547904, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:07:26:768.189 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848879547904, Pid 111888]: Processing scm::EvTcpDeleteSession in state Idle 2025-09-01 Mon 02:07:26:768.255 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848879547904, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:26:768.703 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848879547904, Pid 111888]: SANDESH: Send FAILED: 1756692446767497 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3992-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Idle successful_connections = 0 pid = 111888 http_port = 8085 start_time = 1756692145743123 collector_name = collector_ip = 10.0.0.254:8086 collector_list= [ [ (*_iter40) = 10.0.0.254:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ComputeCpuStateTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 5 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 34 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 50 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 75 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterControlStatsTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 10 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 15 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 112 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 29 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 107, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 231, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:07:26:768.752 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848879547904, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:07:27:134.195 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:27:134.306 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: SANDESH: Send FAILED: 1756692447134125 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.17 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:27:134.338 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:27:134.476 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848858555904, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:27:134.545 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848886536832, Pid 111888]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.17 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:07:27:134.615 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848858555904, Pid 111888]: SANDESH: Send FAILED: 1756692447134423 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.17 state = Connect event = Connect Event local_port = 38967 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:27:134.735 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848858555904, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:27:134.817 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848854357504, Pid 111888]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:07:27:135.151 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848866952704, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:27:135.246 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848866952704, Pid 111888]: SANDESH: Send FAILED: 1756692447135018 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.17 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:27:135.273 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848866952704, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:27:135.299 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848866952704, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:27:135.334 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848866952704, Pid 111888]: SANDESH: Send FAILED: 1756692447135069 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.17 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:27:135.354 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848866952704, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:28:602.428 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848886536832, Pid 111888]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.20.0.17:8085::10.20.0.17:43636 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 255 2025-09-01 Mon 02:07:28:616.360 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848886536832, Pid 111888]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.20.0.17:8085::10.20.0.17:43640 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 255 2025-09-01 Mon 02:07:38:866.603 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848886536832, Pid 111888]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.0.0.254:8085::10.0.0.254:55684 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 255 2025-09-01 Mon 02:07:42:447.289 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:42:447.455 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: SANDESH: Send FAILED: 1756692462447200 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.17 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:42:447.495 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:42:447.667 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848866952704, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:42:447.671 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848886536832, Pid 111888]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.17 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:07:42:447.791 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848866952704, Pid 111888]: SANDESH: Send FAILED: 1756692462447588 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.17 state = Connect event = Connect Event local_port = 43875 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:42:447.906 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848866952704, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:42:447.980 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:07:42:448.232 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848854357504, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:42:448.355 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848854357504, Pid 111888]: SANDESH: Send FAILED: 1756692462448089 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.17 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:42:448.383 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848854357504, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:42:448.408 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848854357504, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:42:448.490 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848854357504, Pid 111888]: SANDESH: Send FAILED: 1756692462448138 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.17 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:42:448.511 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848854357504, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:51:597.648 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848886536832, Pid 111888]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.0.0.254:8085::10.0.0.254:37846 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 255 2025-09-01 Mon 02:07:55:751.566 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848854357504, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:55:752.174 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848854357504, Pid 111888]: SANDESH: Send FAILED: 1756692475751217 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3992-1:Compute:contrail-vrouter-agent:0 sm_queue_count = 0 sm_stats= [ ev_stats= [ [ [ event = scm::EvCollectorUpdate enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvIdleHoldTimerExpired enqueues = 14 dequeues = 14 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvSandeshSend enqueues = 242 dequeues = 242 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvStart enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpConnectFail enqueues = 15 dequeues = 15 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpDeleteSession enqueues = 15 dequeues = 15 enqueue_fails = 0 dequeue_fails = 0 ], [ event = enqueues = 288 dequeues = 288 enqueue_fails = 0 dequeue_fails = 0 ], ] ] state = Idle state_since = 1756692446767393 last_state = Idle last_event = scm::EvSandeshSend last_event_at = 1756692462448404 ] max_sm_queue_count = 6 ] 2025-09-01 Mon 02:07:55:752.261 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848854357504, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:07:55:752.301 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848854357504, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:55:753.045 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848854357504, Pid 111888]: SANDESH: Send FAILED: 1756692475751425 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3992-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Idle successful_connections = 0 pid = 111888 http_port = 8085 start_time = 1756692145743123 collector_name = collector_ip = 10.0.0.254:8086 collector_list= [ [ (*_iter40) = 10.0.0.254:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ComputeCpuStateTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 5 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 35 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 52 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 81 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterControlStatsTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 10 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 15 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 120 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 31 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 115, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 242, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:07:55:753.134 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848854357504, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:07:55:756.979 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848862754304, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:55:757.433 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848862754304, Pid 111888]: SANDESH: Send FAILED: 1756692475756883 [SYS_NOTICE]: VrouterStats: data= [ name = cn-jenkins-deploy-platform-ansible-os-3992-1 in_tpkts = 159 in_bytes = 6846 out_tpkts = 176 out_bytes = 8192 raw_vhost_stats= [ name = vhost0 in_pkts = 159 in_bytes = 6846 out_pkts = 176 out_bytes = 8192 speed = -1 duplexity = -1 ] raw_vhost_drop_stats= [ ds_discard = 0 ds_pull = 0 ds_invalid_if = 0 ds_invalid_arp = 0 ds_trap_no_if = 0 ds_nowhere_to_go = 0 ds_flow_queue_limit_exceeded = 0 ds_flow_no_memory = 0 ds_flow_invalid_protocol = 0 ds_flow_nat_no_rflow = 0 ds_flow_action_drop = 0 ds_flow_action_invalid = 0 ds_flow_unusable = 0 ds_flow_table_full = 0 ds_interface_tx_discard = 0 ds_interface_drop = 0 ds_duplicated = 0 ds_push = 0 ds_ttl_exceeded = 0 ds_invalid_nh = 0 ds_invalid_label = 0 ds_invalid_protocol = 0 ds_interface_rx_discard = 0 ds_invalid_mcast_source = 0 ds_head_alloc_fail = 0 ds_pcow_fail = 0 ds_mcast_clone_fail = 0 ds_rewrite_fail = 0 ds_misc = 0 ds_invalid_packet = 0 ds_cksum_err = 0 ds_no_fmd = 0 ds_invalid_vnid = 0 ds_frag_err = 0 ds_invalid_source = 0 ds_mcast_df_bit = 0 ds_l2_no_route = 0 ds_vlan_fwd_tx = 0 ds_vlan_fwd_enq = 0 ds_drop_new_flow = 0 ds_flow_evict = 0 ds_trap_original = 0 ds_no_memory = 0 ds_fragment_queue_fail = 0 ds_no_frag_entry = 0 ds_icmp_error = 0 ds_clone_fail = 0 ds_drop_pkts = 0 ] phy_if_5min_usage= [ [ [ name = ens6 in_bandwidth_usage = 45 out_bandwidth_usage = 157 ], ] ] raw_drop_stats= [ ds_discard = 0 ds_pull = 0 ds_invalid_if = 0 ds_invalid_arp = 0 ds_trap_no_if = 0 ds_nowhere_to_go = 0 ds_flow_queue_limit_exceeded = 0 ds_flow_no_memory = 0 ds_flow_invalid_protocol = 0 ds_flow_nat_no_rflow = 0 ds_flow_action_drop = 0 ds_flow_action_invalid = 0 ds_flow_unusable = 0 ds_flow_table_full = 0 ds_interface_tx_discard = 0 ds_interface_drop = 0 ds_duplicated = 0 ds_push = 0 ds_ttl_exceeded = 0 ds_invalid_nh = 0 ds_invalid_label = 0 ds_invalid_protocol = 0 ds_interface_rx_discard = 0 ds_invalid_mcast_source = 0 ds_head_alloc_fail = 0 ds_pcow_fail = 0 ds_mcast_clone_fail = 0 ds_rewrite_fail = 0 ds_misc = 0 ds_invalid_packet = 0 ds_cksum_err = 0 ds_no_fmd = 0 ds_invalid_vnid = 0 ds_frag_err = 0 ds_invalid_source = 0 ds_mcast_df_bit = 0 ds_l2_no_route = 0 ds_vlan_fwd_tx = 0 ds_vlan_fwd_enq = 0 ds_drop_new_flow = 0 ds_flow_evict = 0 ds_trap_original = 0 ds_no_memory = 0 ds_fragment_queue_fail = 0 ds_no_frag_entry = 0 ds_icmp_error = 0 ds_clone_fail = 0 ds_drop_pkts = 0 ] total_in_bandwidth_utilization = 4.87891e-17 total_out_bandwidth_utilization = 4.8247e-16 flow_rate= [ added_flows = 0 max_flow_adds_per_second = 0 min_flow_adds_per_second = 0 deleted_flows = 0 max_flow_deletes_per_second = 0 min_flow_deletes_per_second = 0 active_flows = 0 hold_flows = 0 ] phy_band_in_bps= [ [ _iter260->first = ens6 _iter260->second = 9, ] ] phy_band_out_bps= [ [ _iter261->first = ens6 _iter261->second = 89, ] ] raw_phy_if_stats= [ [ _iter262->first = ens6 [ in_pkts = 23 in_bytes = 1766 out_pkts = 159 out_bytes = 6846 ], ] ] raw_phy_if_drop_stats= [ [ _iter263->first = ens6 [ ds_discard = 0 ds_pull = 0 ds_invalid_if = 0 ds_invalid_arp = 0 ds_trap_no_if = 0 ds_nowhere_to_go = 0 ds_flow_queue_limit_exceeded = 0 ds_flow_no_memory = 0 ds_flow_invalid_protocol = 0 ds_flow_nat_no_rflow = 0 ds_flow_action_drop = 0 ds_flow_action_invalid = 0 ds_flow_unusable = 0 ds_flow_table_full = 0 ds_interface_tx_discard = 0 ds_interface_drop = 0 ds_duplicated = 0 ds_push = 0 ds_ttl_exceeded = 0 ds_invalid_nh = 0 ds_invalid_label = 0 ds_invalid_protocol = 0 ds_interface_rx_discard = 0 ds_invalid_mcast_source = 0 ds_head_alloc_fail = 0 ds_pcow_fail = 0 ds_mcast_clone_fail = 0 ds_rewrite_fail = 0 ds_misc = 0 ds_invalid_packet = 0 ds_cksum_err = 0 ds_no_fmd = 0 ds_invalid_vnid = 0 ds_frag_err = 0 ds_invalid_source = 0 ds_mcast_df_bit = 0 ds_l2_no_route = 0 ds_vlan_fwd_tx = 0 ds_vlan_fwd_enq = 0 ds_drop_new_flow = 0 ds_flow_evict = 0 ds_trap_original = 0 ds_no_memory = 0 ds_fragment_queue_fail = 0 ds_no_frag_entry = 0 ds_icmp_error = 0 ds_clone_fail = 0 ds_drop_pkts = 0 ], ] ] ] 2025-09-01 Mon 02:07:55:757.498 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848862754304, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: VrouterStats 2025-09-01 Mon 02:07:55:757.556 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848862754304, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:55:757.635 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848862754304, Pid 111888]: SANDESH: Send FAILED: 1756692475757029 [SYS_NOTICE]: VrouterControlStatsTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3992-1 raw_xmpp_stats= [ [ _iter306->first = 10.20.0.17 [ reconnects = 0 in_msgs = 0 out_msgs = 0 ], ] ] raw_ifmap_stats= [ [ _iter307->first = link_delete_parse_errors _iter307->second = 0, _iter307->first = link_update_parse_errors _iter307->second = 0, _iter307->first = node_delete_parse_errors _iter307->second = 0, _iter307->first = node_update_parse_errors _iter307->second = 0, ] ] raw_rt_table_size= [ [ _iter308->first = default-domain:default-project:ip-fabric:__default__ [ inet4_unicast = 4 inet4_multicast = 0 evpn = 0 bridge = 2 inet6_unicast = 2 ], _iter308->first = default-domain:default-project:ip-fabric:ip-fabric [ inet4_unicast = 2 inet4_multicast = 0 evpn = 1 bridge = 2 inet6_unicast = 0 ], ] ] ] 2025-09-01 Mon 02:07:55:757.722 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848862754304, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: VrouterControlStatsTrace 2025-09-01 Mon 02:07:56:767.595 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848854357504, Pid 111888]: Processing scm::EvIdleHoldTimerExpired in state Idle 2025-09-01 Mon 02:07:56:805.168 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848854357504, Pid 111888]: Connect : Start Connect timer 10.0.0.254:8086 2025-09-01 Mon 02:07:56:805.267 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848886536832, Pid 111888]: -(-1) OnSessionEvent TCP Connect Failed 2025-09-01 Mon 02:07:56:805.283 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848854357504, Pid 111888]: Processing scm::EvSandeshSend in state Connect 2025-09-01 Mon 02:07:56:805.357 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848886536832, Pid 111888]: -(-1):EvTcpConnectFail 2025-09-01 Mon 02:07:56:805.388 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848854357504, Pid 111888]: SANDESH: Send FAILED: 1756692476804986 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-3992-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.17 server_addrs= [ [ (*_iter6) = 10.20.0.17:5269, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.17 server_addrs= [ [ (*_iter6) = 10.20.0.17:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.254:8086, ] ] status = Initializing description = Idle : EvIdleHoldTimerExpired -> Connect ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.17, XMPP:dns-server:10.20.0.17, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:07:56:805.446 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848854357504, Pid 111888]: Wrong state: Connect for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:07:56:805.456 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848854357504, Pid 111888]: Processing scm::EvSandeshSend in state Connect 2025-09-01 Mon 02:07:56:805.598 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848854357504, Pid 111888]: SANDESH: Send FAILED: 1756692476805265 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3992-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Connect successful_connections = 0 pid = 111888 http_port = 8085 start_time = 1756692145743123 collector_name = collector_ip = 10.0.0.254:8086 collector_list= [ [ (*_iter40) = 10.0.0.254:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ComputeCpuStateTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 5 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 35 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 54 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 81 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterControlStatsTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 11 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 16 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 120 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 31 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 115, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 246, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:07:56:805.622 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848854357504, Pid 111888]: Wrong state: Connect for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:07:56:805.633 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848854357504, Pid 111888]: Processing scm::EvTcpConnectFail in state Connect 2025-09-01 Mon 02:07:56:805.640 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848854357504, Pid 111888]: Connect : EvTcpConnectFail 2025-09-01 Mon 02:07:56:805.697 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848854357504, Pid 111888]: TCP [SYS_ERR]: TcpSessionMessageLog: Session -(-1) > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:07:56:805.730 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848854357504, Pid 111888]: Idle 2025-09-01 Mon 02:07:56:805.793 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848854357504, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:56:805.811 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848854357504, Pid 111888]: SANDESH: Send FAILED: 1756692476805677 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-3992-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.17 server_addrs= [ [ (*_iter6) = 10.20.0.17:5269, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.17 server_addrs= [ [ (*_iter6) = 10.20.0.17:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.17, XMPP:dns-server:10.20.0.17, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:07:56:805.822 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848854357504, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:07:56:805.830 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848854357504, Pid 111888]: Processing scm::EvTcpDeleteSession in state Idle 2025-09-01 Mon 02:07:56:805.859 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848854357504, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:56:805.975 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848854357504, Pid 111888]: SANDESH: Send FAILED: 1756692476805782 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3992-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Idle successful_connections = 0 pid = 111888 http_port = 8085 start_time = 1756692145743123 collector_name = collector_ip = 10.0.0.254:8086 collector_list= [ [ (*_iter40) = 10.0.0.254:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ComputeCpuStateTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 5 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 36 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 55 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 82 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterControlStatsTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 11 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 16 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 120 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 31 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 116, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 248, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:07:56:805.991 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848854357504, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:07:58:455.351 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848858555904, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:58:455.520 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848858555904, Pid 111888]: SANDESH: Send FAILED: 1756692478455289 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.17 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:58:455.591 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848858555904, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:58:455.681 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848886536832, Pid 111888]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.17 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:07:58:455.790 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848862754304, Pid 111888]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:07:58:455.682 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:58:456.007 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: SANDESH: Send FAILED: 1756692478455629 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.17 state = Connect event = Connect Event local_port = 43223 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:58:456.029 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:58:456.050 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:58:456.076 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: SANDESH: Send FAILED: 1756692478455866 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.17 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:58:456.113 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:07:58:456.131 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:07:58:456.155 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: SANDESH: Send FAILED: 1756692478455874 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.17 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:07:58:456.170 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:04:625.268 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848886536832, Pid 111888]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.0.0.254:8085::10.0.0.254:54108 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 255 2025-09-01 Mon 02:08:14:308.541 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:14:309.092 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: SANDESH: Send FAILED: 1756692494308475 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.17 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:14:309.123 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:14:309.137 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:14:309.152 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: SANDESH: Send FAILED: 1756692494308736 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.17 state = Connect event = Connect Event local_port = 39943 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:14:309.163 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:14:308.848 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848886536832, Pid 111888]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.17 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:08:14:309.406 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:08:14:309.471 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:14:309.492 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: SANDESH: Send FAILED: 1756692494309455 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.17 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:14:309.500 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:14:309.509 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:14:309.557 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: SANDESH: Send FAILED: 1756692494309462 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.17 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:14:309.566 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:17:691.750 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848886536832, Pid 111888]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.0.0.254:8085::10.0.0.254:55870 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 255 2025-09-01 Mon 02:08:25:751.775 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:25:751.871 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: SANDESH: Send FAILED: 1756692505751602 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3992-1:Compute:contrail-vrouter-agent:0 sm_queue_count = 0 sm_stats= [ ev_stats= [ [ [ event = scm::EvCollectorUpdate enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvIdleHoldTimerExpired enqueues = 15 dequeues = 15 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvSandeshSend enqueues = 258 dequeues = 258 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvStart enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpConnectFail enqueues = 16 dequeues = 16 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpDeleteSession enqueues = 16 dequeues = 16 enqueue_fails = 0 dequeue_fails = 0 ], [ event = enqueues = 307 dequeues = 307 enqueue_fails = 0 dequeue_fails = 0 ], ] ] state = Idle state_since = 1756692476805729 last_state = Idle last_event = scm::EvSandeshSend last_event_at = 1756692494309508 ] max_sm_queue_count = 6 ] 2025-09-01 Mon 02:08:25:751.887 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:08:25:751.898 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:25:752.011 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: SANDESH: Send FAILED: 1756692505751728 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3992-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Idle successful_connections = 0 pid = 111888 http_port = 8085 start_time = 1756692145743123 collector_name = collector_ip = 10.0.0.254:8086 collector_list= [ [ (*_iter40) = 10.0.0.254:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ComputeCpuStateTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 5 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 37 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 56 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 86 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterControlStatsTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 11 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 16 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 128 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 33 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 122, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 258, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:08:25:752.033 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:08:25:757.138 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848879547904, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:25:757.223 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848879547904, Pid 111888]: SANDESH: Send FAILED: 1756692505757112 [SYS_NOTICE]: VrouterStats: data= [ name = cn-jenkins-deploy-platform-ansible-os-3992-1 cpu_info= [ num_cpu = 8 sys_mem_info= [ total = 32 used = 0 free = 0 buffers = 0 cached = 510398136 node_type = ] meminfo= [ virt = 32561 peakvirt = 0 res = 0 ] cpuload= [ one_min_avg = 0.1725 five_min_avg = 0.22 fifteen_min_avg = 0.22625 ] cpu_share = 0.41875 ] ] 2025-09-01 Mon 02:08:25:757.243 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848879547904, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: VrouterStats 2025-09-01 Mon 02:08:25:757.252 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848879547904, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:25:757.266 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848879547904, Pid 111888]: SANDESH: Send FAILED: 1756692505757124 [SYS_NOTICE]: ComputeCpuStateTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3992-1 cpu_info= [ [ [ mem_virt = 32561 cpu_share = 0.41875 used_sys_mem = 3784569160 one_min_cpuload = 0.1725 mem_res = 0 ], ] ] ] 2025-09-01 Mon 02:08:25:757.272 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848879547904, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: ComputeCpuStateTrace 2025-09-01 Mon 02:08:25:757.281 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848879547904, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:25:757.372 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848879547904, Pid 111888]: SANDESH: Send FAILED: 1756692505757146 [SYS_NOTICE]: VrouterStats: data= [ name = cn-jenkins-deploy-platform-ansible-os-3992-1 in_tpkts = 162 in_bytes = 6972 out_tpkts = 180 out_bytes = 8388 raw_vhost_stats= [ name = vhost0 in_pkts = 162 in_bytes = 6972 out_pkts = 180 out_bytes = 8388 speed = -1 duplexity = -1 ] raw_vhost_drop_stats= [ ds_discard = 0 ds_pull = 0 ds_invalid_if = 0 ds_invalid_arp = 0 ds_trap_no_if = 0 ds_nowhere_to_go = 0 ds_flow_queue_limit_exceeded = 0 ds_flow_no_memory = 0 ds_flow_invalid_protocol = 0 ds_flow_nat_no_rflow = 0 ds_flow_action_drop = 0 ds_flow_action_invalid = 0 ds_flow_unusable = 0 ds_flow_table_full = 0 ds_interface_tx_discard = 0 ds_interface_drop = 0 ds_duplicated = 0 ds_push = 0 ds_ttl_exceeded = 0 ds_invalid_nh = 0 ds_invalid_label = 0 ds_invalid_protocol = 0 ds_interface_rx_discard = 0 ds_invalid_mcast_source = 0 ds_head_alloc_fail = 0 ds_pcow_fail = 0 ds_mcast_clone_fail = 0 ds_rewrite_fail = 0 ds_misc = 0 ds_invalid_packet = 0 ds_cksum_err = 0 ds_no_fmd = 0 ds_invalid_vnid = 0 ds_frag_err = 0 ds_invalid_source = 0 ds_mcast_df_bit = 0 ds_l2_no_route = 0 ds_vlan_fwd_tx = 0 ds_vlan_fwd_enq = 0 ds_drop_new_flow = 0 ds_flow_evict = 0 ds_trap_original = 0 ds_no_memory = 0 ds_fragment_queue_fail = 0 ds_no_frag_entry = 0 ds_icmp_error = 0 ds_clone_fail = 0 ds_drop_pkts = 0 ] raw_drop_stats= [ ds_discard = 0 ds_pull = 0 ds_invalid_if = 0 ds_invalid_arp = 0 ds_trap_no_if = 0 ds_nowhere_to_go = 0 ds_flow_queue_limit_exceeded = 0 ds_flow_no_memory = 0 ds_flow_invalid_protocol = 0 ds_flow_nat_no_rflow = 0 ds_flow_action_drop = 0 ds_flow_action_invalid = 0 ds_flow_unusable = 0 ds_flow_table_full = 0 ds_interface_tx_discard = 0 ds_interface_drop = 0 ds_duplicated = 0 ds_push = 0 ds_ttl_exceeded = 0 ds_invalid_nh = 0 ds_invalid_label = 0 ds_invalid_protocol = 0 ds_interface_rx_discard = 0 ds_invalid_mcast_source = 0 ds_head_alloc_fail = 0 ds_pcow_fail = 0 ds_mcast_clone_fail = 0 ds_rewrite_fail = 0 ds_misc = 0 ds_invalid_packet = 0 ds_cksum_err = 0 ds_no_fmd = 0 ds_invalid_vnid = 0 ds_frag_err = 0 ds_invalid_source = 0 ds_mcast_df_bit = 0 ds_l2_no_route = 0 ds_vlan_fwd_tx = 0 ds_vlan_fwd_enq = 0 ds_drop_new_flow = 0 ds_flow_evict = 0 ds_trap_original = 0 ds_no_memory = 0 ds_fragment_queue_fail = 0 ds_no_frag_entry = 0 ds_icmp_error = 0 ds_clone_fail = 0 ds_drop_pkts = 0 ] flow_rate= [ added_flows = 0 max_flow_adds_per_second = 0 min_flow_adds_per_second = 0 deleted_flows = 0 max_flow_deletes_per_second = 0 min_flow_deletes_per_second = 0 active_flows = 0 hold_flows = 0 ] raw_phy_if_stats= [ [ _iter262->first = ens6 [ in_pkts = 24 in_bytes = 1836 out_pkts = 162 out_bytes = 6972 ], ] ] raw_phy_if_drop_stats= [ [ _iter263->first = ens6 [ ds_discard = 0 ds_pull = 0 ds_invalid_if = 0 ds_invalid_arp = 0 ds_trap_no_if = 0 ds_nowhere_to_go = 0 ds_flow_queue_limit_exceeded = 0 ds_flow_no_memory = 0 ds_flow_invalid_protocol = 0 ds_flow_nat_no_rflow = 0 ds_flow_action_drop = 0 ds_flow_action_invalid = 0 ds_flow_unusable = 0 ds_flow_table_full = 0 ds_interface_tx_discard = 0 ds_interface_drop = 0 ds_duplicated = 0 ds_push = 0 ds_ttl_exceeded = 0 ds_invalid_nh = 0 ds_invalid_label = 0 ds_invalid_protocol = 0 ds_interface_rx_discard = 0 ds_invalid_mcast_source = 0 ds_head_alloc_fail = 0 ds_pcow_fail = 0 ds_mcast_clone_fail = 0 ds_rewrite_fail = 0 ds_misc = 0 ds_invalid_packet = 0 ds_cksum_err = 0 ds_no_fmd = 0 ds_invalid_vnid = 0 ds_frag_err = 0 ds_invalid_source = 0 ds_mcast_df_bit = 0 ds_l2_no_route = 0 ds_vlan_fwd_tx = 0 ds_vlan_fwd_enq = 0 ds_drop_new_flow = 0 ds_flow_evict = 0 ds_trap_original = 0 ds_no_memory = 0 ds_fragment_queue_fail = 0 ds_no_frag_entry = 0 ds_icmp_error = 0 ds_clone_fail = 0 ds_drop_pkts = 0 ], ] ] ] 2025-09-01 Mon 02:08:25:757.384 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848879547904, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: VrouterStats 2025-09-01 Mon 02:08:25:757.396 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848879547904, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:25:757.418 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848879547904, Pid 111888]: SANDESH: Send FAILED: 1756692505757180 [SYS_NOTICE]: VrouterControlStatsTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3992-1 raw_xmpp_stats= [ [ _iter306->first = 10.20.0.17 [ reconnects = 0 in_msgs = 0 out_msgs = 0 ], ] ] raw_ifmap_stats= [ [ _iter307->first = link_delete_parse_errors _iter307->second = 0, _iter307->first = link_update_parse_errors _iter307->second = 0, _iter307->first = node_delete_parse_errors _iter307->second = 0, _iter307->first = node_update_parse_errors _iter307->second = 0, ] ] raw_rt_table_size= [ [ _iter308->first = default-domain:default-project:ip-fabric:__default__ [ inet4_unicast = 4 inet4_multicast = 0 evpn = 0 bridge = 2 inet6_unicast = 2 ], _iter308->first = default-domain:default-project:ip-fabric:ip-fabric [ inet4_unicast = 2 inet4_multicast = 0 evpn = 1 bridge = 2 inet6_unicast = 0 ], ] ] ] 2025-09-01 Mon 02:08:25:757.424 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848879547904, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: VrouterControlStatsTrace 2025-09-01 Mon 02:08:26:805.901 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848862754304, Pid 111888]: Processing scm::EvIdleHoldTimerExpired in state Idle 2025-09-01 Mon 02:08:26:806.298 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848862754304, Pid 111888]: Connect : Start Connect timer 10.0.0.254:8086 2025-09-01 Mon 02:08:26:806.326 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848886536832, Pid 111888]: -(-1) OnSessionEvent TCP Connect Failed 2025-09-01 Mon 02:08:26:806.371 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848886536832, Pid 111888]: -(-1):EvTcpConnectFail 2025-09-01 Mon 02:08:26:806.445 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848862754304, Pid 111888]: Processing scm::EvSandeshSend in state Connect 2025-09-01 Mon 02:08:26:806.491 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848862754304, Pid 111888]: SANDESH: Send FAILED: 1756692506806027 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-3992-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.17 server_addrs= [ [ (*_iter6) = 10.20.0.17:5269, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.17 server_addrs= [ [ (*_iter6) = 10.20.0.17:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.254:8086, ] ] status = Initializing description = Idle : EvIdleHoldTimerExpired -> Connect ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.17, XMPP:dns-server:10.20.0.17, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:08:26:806.603 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848862754304, Pid 111888]: Wrong state: Connect for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:08:26:806.623 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848862754304, Pid 111888]: Processing scm::EvTcpConnectFail in state Connect 2025-09-01 Mon 02:08:26:806.635 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848862754304, Pid 111888]: Connect : EvTcpConnectFail 2025-09-01 Mon 02:08:26:806.700 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848862754304, Pid 111888]: TCP [SYS_ERR]: TcpSessionMessageLog: Session -(-1) > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:08:26:806.863 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848862754304, Pid 111888]: Idle 2025-09-01 Mon 02:08:26:806.960 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848862754304, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:26:807.250 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848862754304, Pid 111888]: SANDESH: Send FAILED: 1756692506806416 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3992-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Connect successful_connections = 0 pid = 111888 http_port = 8085 start_time = 1756692145743123 collector_name = collector_ip = 10.0.0.254:8086 collector_list= [ [ (*_iter40) = 10.0.0.254:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ComputeCpuStateTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 6 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 37 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 58 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 86 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterControlStatsTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 12 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 18 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 128 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 33 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 122, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 264, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:08:26:807.283 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848862754304, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:08:26:807.301 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848862754304, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:26:807.330 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848862754304, Pid 111888]: SANDESH: Send FAILED: 1756692506806668 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-3992-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.17 server_addrs= [ [ (*_iter6) = 10.20.0.17:5269, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.17 server_addrs= [ [ (*_iter6) = 10.20.0.17:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.17, XMPP:dns-server:10.20.0.17, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:08:26:807.341 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848862754304, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:08:26:807.353 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848862754304, Pid 111888]: Processing scm::EvTcpDeleteSession in state Idle 2025-09-01 Mon 02:08:26:807.395 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848862754304, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:26:807.663 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848862754304, Pid 111888]: SANDESH: Send FAILED: 1756692506806942 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3992-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Idle successful_connections = 0 pid = 111888 http_port = 8085 start_time = 1756692145743123 collector_name = collector_ip = 10.0.0.254:8086 collector_list= [ [ (*_iter40) = 10.0.0.254:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ComputeCpuStateTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 6 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 38 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 58 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 87 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterControlStatsTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 12 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 18 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 128 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 33 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 123, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 265, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:08:26:807.695 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848862754304, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:08:29:563.088 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848886536832, Pid 111888]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.20.0.17:8085::10.20.0.17:57420 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 255 2025-09-01 Mon 02:08:29:580.617 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848886536832, Pid 111888]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.20.0.17:8085::10.20.0.17:57434 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 255 2025-09-01 Mon 02:08:30:316.157 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:30:316.217 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: SANDESH: Send FAILED: 1756692510316122 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.17 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:30:316.229 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:30:316.517 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848862754304, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:30:316.641 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848862754304, Pid 111888]: SANDESH: Send FAILED: 1756692510316454 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.17 state = Connect event = Connect Event local_port = 39943 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:30:316.674 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848862754304, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:30:316.576 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848886536832, Pid 111888]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.17 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:08:30:316.956 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848854357504, Pid 111888]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:08:30:317.197 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:30:317.311 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: SANDESH: Send FAILED: 1756692510317061 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.17 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:30:317.348 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:30:317.386 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:30:317.473 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: SANDESH: Send FAILED: 1756692510317121 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.17 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:30:317.503 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:30:486.215 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848886536832, Pid 111888]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.0.0.254:8085::10.0.0.254:48664 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 255 2025-09-01 Mon 02:08:42:659.739 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848854357504, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:42:659.838 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848854357504, Pid 111888]: SANDESH: Send FAILED: 1756692522659687 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.17 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:42:659.860 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848854357504, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:42:659.919 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848879547904, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:42:659.967 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848879547904, Pid 111888]: SANDESH: Send FAILED: 1756692522659890 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.17 state = Connect event = Connect Event local_port = 45321 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:42:659.982 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848879547904, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:42:659.940 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848886536832, Pid 111888]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.17 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:08:42:660.060 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848879547904, Pid 111888]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:08:42:660.127 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:42:660.210 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: SANDESH: Send FAILED: 1756692522660088 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.17 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:42:660.235 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:42:660.258 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:42:660.284 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: SANDESH: Send FAILED: 1756692522660091 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.17 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:42:660.300 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:43:165.499 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848886536832, Pid 111888]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.0.0.254:8085::10.0.0.254:47812 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 255 2025-09-01 Mon 02:08:55:752.319 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:55:752.507 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: SANDESH: Send FAILED: 1756692535752001 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3992-1:Compute:contrail-vrouter-agent:0 sm_queue_count = 0 sm_stats= [ ev_stats= [ [ [ event = scm::EvCollectorUpdate enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvIdleHoldTimerExpired enqueues = 16 dequeues = 16 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvSandeshSend enqueues = 276 dequeues = 276 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvStart enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpConnectFail enqueues = 17 dequeues = 17 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpDeleteSession enqueues = 17 dequeues = 17 enqueue_fails = 0 dequeue_fails = 0 ], [ event = enqueues = 328 dequeues = 328 enqueue_fails = 0 dequeue_fails = 0 ], ] ] state = Idle state_since = 1756692506806861 last_state = Idle last_event = scm::EvSandeshSend last_event_at = 1756692522660254 ] max_sm_queue_count = 6 ] 2025-09-01 Mon 02:08:55:752.571 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:08:55:752.599 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:55:753.090 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: SANDESH: Send FAILED: 1756692535752214 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3992-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Idle successful_connections = 0 pid = 111888 http_port = 8085 start_time = 1756692145743123 collector_name = collector_ip = 10.0.0.254:8086 collector_list= [ [ (*_iter40) = 10.0.0.254:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ComputeCpuStateTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 6 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 39 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 60 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 93 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterControlStatsTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 12 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 18 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 136 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 35 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 131, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 276, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:08:55:753.160 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:08:55:757.567 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848866952704, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:55:757.950 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848866952704, Pid 111888]: SANDESH: Send FAILED: 1756692535757470 [SYS_NOTICE]: VrouterStats: data= [ name = cn-jenkins-deploy-platform-ansible-os-3992-1 in_tpkts = 165 in_bytes = 7098 out_tpkts = 183 out_bytes = 8514 raw_vhost_stats= [ name = vhost0 in_pkts = 165 in_bytes = 7098 out_pkts = 183 out_bytes = 8514 speed = -1 duplexity = -1 ] raw_vhost_drop_stats= [ ds_discard = 0 ds_pull = 0 ds_invalid_if = 0 ds_invalid_arp = 0 ds_trap_no_if = 0 ds_nowhere_to_go = 0 ds_flow_queue_limit_exceeded = 0 ds_flow_no_memory = 0 ds_flow_invalid_protocol = 0 ds_flow_nat_no_rflow = 0 ds_flow_action_drop = 0 ds_flow_action_invalid = 0 ds_flow_unusable = 0 ds_flow_table_full = 0 ds_interface_tx_discard = 0 ds_interface_drop = 0 ds_duplicated = 0 ds_push = 0 ds_ttl_exceeded = 0 ds_invalid_nh = 0 ds_invalid_label = 0 ds_invalid_protocol = 0 ds_interface_rx_discard = 0 ds_invalid_mcast_source = 0 ds_head_alloc_fail = 0 ds_pcow_fail = 0 ds_mcast_clone_fail = 0 ds_rewrite_fail = 0 ds_misc = 0 ds_invalid_packet = 0 ds_cksum_err = 0 ds_no_fmd = 0 ds_invalid_vnid = 0 ds_frag_err = 0 ds_invalid_source = 0 ds_mcast_df_bit = 0 ds_l2_no_route = 0 ds_vlan_fwd_tx = 0 ds_vlan_fwd_enq = 0 ds_drop_new_flow = 0 ds_flow_evict = 0 ds_trap_original = 0 ds_no_memory = 0 ds_fragment_queue_fail = 0 ds_no_frag_entry = 0 ds_icmp_error = 0 ds_clone_fail = 0 ds_drop_pkts = 0 ] raw_drop_stats= [ ds_discard = 0 ds_pull = 0 ds_invalid_if = 0 ds_invalid_arp = 0 ds_trap_no_if = 0 ds_nowhere_to_go = 0 ds_flow_queue_limit_exceeded = 0 ds_flow_no_memory = 0 ds_flow_invalid_protocol = 0 ds_flow_nat_no_rflow = 0 ds_flow_action_drop = 0 ds_flow_action_invalid = 0 ds_flow_unusable = 0 ds_flow_table_full = 0 ds_interface_tx_discard = 0 ds_interface_drop = 0 ds_duplicated = 0 ds_push = 0 ds_ttl_exceeded = 0 ds_invalid_nh = 0 ds_invalid_label = 0 ds_invalid_protocol = 0 ds_interface_rx_discard = 0 ds_invalid_mcast_source = 0 ds_head_alloc_fail = 0 ds_pcow_fail = 0 ds_mcast_clone_fail = 0 ds_rewrite_fail = 0 ds_misc = 0 ds_invalid_packet = 0 ds_cksum_err = 0 ds_no_fmd = 0 ds_invalid_vnid = 0 ds_frag_err = 0 ds_invalid_source = 0 ds_mcast_df_bit = 0 ds_l2_no_route = 0 ds_vlan_fwd_tx = 0 ds_vlan_fwd_enq = 0 ds_drop_new_flow = 0 ds_flow_evict = 0 ds_trap_original = 0 ds_no_memory = 0 ds_fragment_queue_fail = 0 ds_no_frag_entry = 0 ds_icmp_error = 0 ds_clone_fail = 0 ds_drop_pkts = 0 ] total_out_bandwidth_utilization = 1.78893e-16 flow_rate= [ added_flows = 0 max_flow_adds_per_second = 0 min_flow_adds_per_second = 0 deleted_flows = 0 max_flow_deletes_per_second = 0 min_flow_deletes_per_second = 0 active_flows = 0 hold_flows = 0 ] phy_band_in_bps= [ [ _iter260->first = ens6 _iter260->second = 9, ] ] phy_band_out_bps= [ [ _iter261->first = ens6 _iter261->second = 33, ] ] raw_phy_if_stats= [ [ _iter262->first = ens6 [ in_pkts = 24 in_bytes = 1836 out_pkts = 165 out_bytes = 7098 ], ] ] raw_phy_if_drop_stats= [ [ _iter263->first = ens6 [ ds_discard = 0 ds_pull = 0 ds_invalid_if = 0 ds_invalid_arp = 0 ds_trap_no_if = 0 ds_nowhere_to_go = 0 ds_flow_queue_limit_exceeded = 0 ds_flow_no_memory = 0 ds_flow_invalid_protocol = 0 ds_flow_nat_no_rflow = 0 ds_flow_action_drop = 0 ds_flow_action_invalid = 0 ds_flow_unusable = 0 ds_flow_table_full = 0 ds_interface_tx_discard = 0 ds_interface_drop = 0 ds_duplicated = 0 ds_push = 0 ds_ttl_exceeded = 0 ds_invalid_nh = 0 ds_invalid_label = 0 ds_invalid_protocol = 0 ds_interface_rx_discard = 0 ds_invalid_mcast_source = 0 ds_head_alloc_fail = 0 ds_pcow_fail = 0 ds_mcast_clone_fail = 0 ds_rewrite_fail = 0 ds_misc = 0 ds_invalid_packet = 0 ds_cksum_err = 0 ds_no_fmd = 0 ds_invalid_vnid = 0 ds_frag_err = 0 ds_invalid_source = 0 ds_mcast_df_bit = 0 ds_l2_no_route = 0 ds_vlan_fwd_tx = 0 ds_vlan_fwd_enq = 0 ds_drop_new_flow = 0 ds_flow_evict = 0 ds_trap_original = 0 ds_no_memory = 0 ds_fragment_queue_fail = 0 ds_no_frag_entry = 0 ds_icmp_error = 0 ds_clone_fail = 0 ds_drop_pkts = 0 ], ] ] ] 2025-09-01 Mon 02:08:55:757.994 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848866952704, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: VrouterStats 2025-09-01 Mon 02:08:55:758.011 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848866952704, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:55:758.052 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848866952704, Pid 111888]: SANDESH: Send FAILED: 1756692535757612 [SYS_NOTICE]: VrouterControlStatsTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3992-1 raw_xmpp_stats= [ [ _iter306->first = 10.20.0.17 [ reconnects = 0 in_msgs = 0 out_msgs = 0 ], ] ] raw_ifmap_stats= [ [ _iter307->first = link_delete_parse_errors _iter307->second = 0, _iter307->first = link_update_parse_errors _iter307->second = 0, _iter307->first = node_delete_parse_errors _iter307->second = 0, _iter307->first = node_update_parse_errors _iter307->second = 0, ] ] raw_rt_table_size= [ [ _iter308->first = default-domain:default-project:ip-fabric:__default__ [ inet4_unicast = 4 inet4_multicast = 0 evpn = 0 bridge = 2 inet6_unicast = 2 ], _iter308->first = default-domain:default-project:ip-fabric:ip-fabric [ inet4_unicast = 2 inet4_multicast = 0 evpn = 1 bridge = 2 inet6_unicast = 0 ], ] ] ] 2025-09-01 Mon 02:08:55:758.067 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848866952704, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: VrouterControlStatsTrace 2025-09-01 Mon 02:08:55:801.759 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848886536832, Pid 111888]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.0.0.254:8085::10.0.0.254:42984 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 255 2025-09-01 Mon 02:08:56:807.021 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: Processing scm::EvIdleHoldTimerExpired in state Idle 2025-09-01 Mon 02:08:56:807.297 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: Connect : Start Connect timer 10.0.0.254:8086 2025-09-01 Mon 02:08:56:807.366 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848886536832, Pid 111888]: -(-1) OnSessionEvent TCP Connect Failed 2025-09-01 Mon 02:08:56:807.408 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: Processing scm::EvSandeshSend in state Connect 2025-09-01 Mon 02:08:56:807.481 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: SANDESH: Send FAILED: 1756692536807117 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-3992-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.17 server_addrs= [ [ (*_iter6) = 10.20.0.17:5269, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.17 server_addrs= [ [ (*_iter6) = 10.20.0.17:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.254:8086, ] ] status = Initializing description = Idle : EvIdleHoldTimerExpired -> Connect ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.17, XMPP:dns-server:10.20.0.17, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:08:56:807.490 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: Wrong state: Connect for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:08:56:807.500 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: Processing scm::EvSandeshSend in state Connect 2025-09-01 Mon 02:08:56:807.604 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848886536832, Pid 111888]: -(-1):EvTcpConnectFail 2025-09-01 Mon 02:08:56:807.651 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: SANDESH: Send FAILED: 1756692536807384 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3992-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Connect successful_connections = 0 pid = 111888 http_port = 8085 start_time = 1756692145743123 collector_name = collector_ip = 10.0.0.254:8086 collector_list= [ [ (*_iter40) = 10.0.0.254:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ComputeCpuStateTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 6 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 39 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 62 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 94 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterControlStatsTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 13 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 19 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 136 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 35 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 132, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 280, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:08:56:807.675 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: Wrong state: Connect for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:08:56:807.686 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: Processing scm::EvTcpConnectFail in state Connect 2025-09-01 Mon 02:08:56:807.696 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: Connect : EvTcpConnectFail 2025-09-01 Mon 02:08:56:807.738 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: TCP [SYS_ERR]: TcpSessionMessageLog: Session -(-1) > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:08:56:807.764 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: Idle 2025-09-01 Mon 02:08:56:807.821 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:56:807.838 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: SANDESH: Send FAILED: 1756692536807717 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-3992-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.17 server_addrs= [ [ (*_iter6) = 10.20.0.17:5269, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.17 server_addrs= [ [ (*_iter6) = 10.20.0.17:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.17, XMPP:dns-server:10.20.0.17, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:08:56:807.845 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:08:56:807.853 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: Processing scm::EvTcpDeleteSession in state Idle 2025-09-01 Mon 02:08:56:807.880 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:56:807.997 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: SANDESH: Send FAILED: 1756692536807808 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3992-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Idle successful_connections = 0 pid = 111888 http_port = 8085 start_time = 1756692145743123 collector_name = collector_ip = 10.0.0.254:8086 collector_list= [ [ (*_iter40) = 10.0.0.254:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ComputeCpuStateTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 6 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 40 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 63 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 95 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterControlStatsTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 13 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 19 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 136 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 35 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 133, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 282, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:08:56:808.014 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:08:59:207.436 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:59:207.511 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: SANDESH: Send FAILED: 1756692539207381 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.17 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:59:207.523 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:59:207.615 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848858555904, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:59:207.670 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848858555904, Pid 111888]: SANDESH: Send FAILED: 1756692539207577 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.17 state = Connect event = Connect Event local_port = 39009 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:59:207.679 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848858555904, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:59:207.682 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848886536832, Pid 111888]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.17 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:08:59:207.929 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:08:59:208.110 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848879547904, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:59:208.153 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848879547904, Pid 111888]: SANDESH: Send FAILED: 1756692539208044 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.17 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:59:208.164 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848879547904, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:08:59:208.190 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848879547904, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:08:59:208.214 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848879547904, Pid 111888]: SANDESH: Send FAILED: 1756692539208098 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.17 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:08:59:208.222 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848879547904, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:08:918.637 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848886536832, Pid 111888]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.0.0.254:8085::10.0.0.254:39432 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 255 2025-09-01 Mon 02:09:11:280.526 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:11:280.685 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: SANDESH: Send FAILED: 1756692551280442 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.17 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:11:280.714 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:11:280.744 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:11:280.783 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848886536832, Pid 111888]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.17 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:09:11:280.797 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: SANDESH: Send FAILED: 1756692551280722 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.17 state = Connect event = Connect Event local_port = 41561 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:11:280.917 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:09:11:281.033 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:11:281.096 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:11:281.161 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: SANDESH: Send FAILED: 1756692551281066 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.17 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:11:281.193 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:11:281.227 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:11:281.273 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: SANDESH: Send FAILED: 1756692551281073 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.17 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:11:281.301 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:21:746.033 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848886536832, Pid 111888]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.0.0.254:8085::10.0.0.254:53586 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 255 2025-09-01 Mon 02:09:25:753.018 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848879547904, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:25:753.299 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848879547904, Pid 111888]: SANDESH: Send FAILED: 1756692565752596 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3992-1:Compute:contrail-vrouter-agent:0 sm_queue_count = 0 sm_stats= [ ev_stats= [ [ [ event = scm::EvCollectorUpdate enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvIdleHoldTimerExpired enqueues = 17 dequeues = 17 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvSandeshSend enqueues = 292 dequeues = 292 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvStart enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpConnectFail enqueues = 18 dequeues = 18 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpDeleteSession enqueues = 18 dequeues = 18 enqueue_fails = 0 dequeue_fails = 0 ], [ event = enqueues = 347 dequeues = 347 enqueue_fails = 0 dequeue_fails = 0 ], ] ] state = Idle state_since = 1756692536807763 last_state = Idle last_event = scm::EvSandeshSend last_event_at = 1756692551281220 ] max_sm_queue_count = 6 ] 2025-09-01 Mon 02:09:25:753.351 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848879547904, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:09:25:753.388 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848879547904, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:25:754.292 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848879547904, Pid 111888]: SANDESH: Send FAILED: 1756692565752885 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3992-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Idle successful_connections = 0 pid = 111888 http_port = 8085 start_time = 1756692145743123 collector_name = collector_ip = 10.0.0.254:8086 collector_list= [ [ (*_iter40) = 10.0.0.254:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ComputeCpuStateTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 6 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 41 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 64 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 99 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterControlStatsTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 13 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 19 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 144 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 37 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 139, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 292, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:09:25:754.606 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848879547904, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:09:25:758.182 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:25:758.404 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: SANDESH: Send FAILED: 1756692565758069 [SYS_NOTICE]: VrouterStats: data= [ name = cn-jenkins-deploy-platform-ansible-os-3992-1 cpu_info= [ num_cpu = 8 sys_mem_info= [ total = 32 used = 0 free = 0 buffers = 0 cached = 510398136 node_type = ] meminfo= [ virt = 32561 peakvirt = 0 res = 0 ] cpuload= [ one_min_avg = 0.1575 five_min_avg = 0.20875 fifteen_min_avg = 0.2225 ] cpu_share = 0.260373 ] ] 2025-09-01 Mon 02:09:25:758.447 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: VrouterStats 2025-09-01 Mon 02:09:25:758.488 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:25:758.585 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: SANDESH: Send FAILED: 1756692565758106 [SYS_NOTICE]: ComputeCpuStateTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3992-1 cpu_info= [ [ [ mem_virt = 32561 cpu_share = 0.260373 used_sys_mem = 3784569160 one_min_cpuload = 0.1575 mem_res = 0 ], ] ] ] 2025-09-01 Mon 02:09:25:758.620 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: ComputeCpuStateTrace 2025-09-01 Mon 02:09:25:758.655 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:25:759.071 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: SANDESH: Send FAILED: 1756692565758161 [SYS_NOTICE]: VrouterStats: data= [ name = cn-jenkins-deploy-platform-ansible-os-3992-1 in_tpkts = 168 in_bytes = 7224 out_tpkts = 186 out_bytes = 8640 raw_vhost_stats= [ name = vhost0 in_pkts = 168 in_bytes = 7224 out_pkts = 186 out_bytes = 8640 speed = -1 duplexity = -1 ] raw_vhost_drop_stats= [ ds_discard = 0 ds_pull = 0 ds_invalid_if = 0 ds_invalid_arp = 0 ds_trap_no_if = 0 ds_nowhere_to_go = 0 ds_flow_queue_limit_exceeded = 0 ds_flow_no_memory = 0 ds_flow_invalid_protocol = 0 ds_flow_nat_no_rflow = 0 ds_flow_action_drop = 0 ds_flow_action_invalid = 0 ds_flow_unusable = 0 ds_flow_table_full = 0 ds_interface_tx_discard = 0 ds_interface_drop = 0 ds_duplicated = 0 ds_push = 0 ds_ttl_exceeded = 0 ds_invalid_nh = 0 ds_invalid_label = 0 ds_invalid_protocol = 0 ds_interface_rx_discard = 0 ds_invalid_mcast_source = 0 ds_head_alloc_fail = 0 ds_pcow_fail = 0 ds_mcast_clone_fail = 0 ds_rewrite_fail = 0 ds_misc = 0 ds_invalid_packet = 0 ds_cksum_err = 0 ds_no_fmd = 0 ds_invalid_vnid = 0 ds_frag_err = 0 ds_invalid_source = 0 ds_mcast_df_bit = 0 ds_l2_no_route = 0 ds_vlan_fwd_tx = 0 ds_vlan_fwd_enq = 0 ds_drop_new_flow = 0 ds_flow_evict = 0 ds_trap_original = 0 ds_no_memory = 0 ds_fragment_queue_fail = 0 ds_no_frag_entry = 0 ds_icmp_error = 0 ds_clone_fail = 0 ds_drop_pkts = 0 ] raw_drop_stats= [ ds_discard = 0 ds_pull = 0 ds_invalid_if = 0 ds_invalid_arp = 0 ds_trap_no_if = 0 ds_nowhere_to_go = 0 ds_flow_queue_limit_exceeded = 0 ds_flow_no_memory = 0 ds_flow_invalid_protocol = 0 ds_flow_nat_no_rflow = 0 ds_flow_action_drop = 0 ds_flow_action_invalid = 0 ds_flow_unusable = 0 ds_flow_table_full = 0 ds_interface_tx_discard = 0 ds_interface_drop = 0 ds_duplicated = 0 ds_push = 0 ds_ttl_exceeded = 0 ds_invalid_nh = 0 ds_invalid_label = 0 ds_invalid_protocol = 0 ds_interface_rx_discard = 0 ds_invalid_mcast_source = 0 ds_head_alloc_fail = 0 ds_pcow_fail = 0 ds_mcast_clone_fail = 0 ds_rewrite_fail = 0 ds_misc = 0 ds_invalid_packet = 0 ds_cksum_err = 0 ds_no_fmd = 0 ds_invalid_vnid = 0 ds_frag_err = 0 ds_invalid_source = 0 ds_mcast_df_bit = 0 ds_l2_no_route = 0 ds_vlan_fwd_tx = 0 ds_vlan_fwd_enq = 0 ds_drop_new_flow = 0 ds_flow_evict = 0 ds_trap_original = 0 ds_no_memory = 0 ds_fragment_queue_fail = 0 ds_no_frag_entry = 0 ds_icmp_error = 0 ds_clone_fail = 0 ds_drop_pkts = 0 ] flow_rate= [ added_flows = 0 max_flow_adds_per_second = 0 min_flow_adds_per_second = 0 deleted_flows = 0 max_flow_deletes_per_second = 0 min_flow_deletes_per_second = 0 active_flows = 0 hold_flows = 0 ] raw_phy_if_stats= [ [ _iter262->first = ens6 [ in_pkts = 24 in_bytes = 1836 out_pkts = 168 out_bytes = 7224 ], ] ] raw_phy_if_drop_stats= [ [ _iter263->first = ens6 [ ds_discard = 0 ds_pull = 0 ds_invalid_if = 0 ds_invalid_arp = 0 ds_trap_no_if = 0 ds_nowhere_to_go = 0 ds_flow_queue_limit_exceeded = 0 ds_flow_no_memory = 0 ds_flow_invalid_protocol = 0 ds_flow_nat_no_rflow = 0 ds_flow_action_drop = 0 ds_flow_action_invalid = 0 ds_flow_unusable = 0 ds_flow_table_full = 0 ds_interface_tx_discard = 0 ds_interface_drop = 0 ds_duplicated = 0 ds_push = 0 ds_ttl_exceeded = 0 ds_invalid_nh = 0 ds_invalid_label = 0 ds_invalid_protocol = 0 ds_interface_rx_discard = 0 ds_invalid_mcast_source = 0 ds_head_alloc_fail = 0 ds_pcow_fail = 0 ds_mcast_clone_fail = 0 ds_rewrite_fail = 0 ds_misc = 0 ds_invalid_packet = 0 ds_cksum_err = 0 ds_no_fmd = 0 ds_invalid_vnid = 0 ds_frag_err = 0 ds_invalid_source = 0 ds_mcast_df_bit = 0 ds_l2_no_route = 0 ds_vlan_fwd_tx = 0 ds_vlan_fwd_enq = 0 ds_drop_new_flow = 0 ds_flow_evict = 0 ds_trap_original = 0 ds_no_memory = 0 ds_fragment_queue_fail = 0 ds_no_frag_entry = 0 ds_icmp_error = 0 ds_clone_fail = 0 ds_drop_pkts = 0 ], ] ] ] 2025-09-01 Mon 02:09:25:759.140 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: VrouterStats 2025-09-01 Mon 02:09:25:759.169 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:25:759.264 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: SANDESH: Send FAILED: 1756692565758230 [SYS_NOTICE]: VrouterControlStatsTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3992-1 raw_xmpp_stats= [ [ _iter306->first = 10.20.0.17 [ reconnects = 0 in_msgs = 0 out_msgs = 0 ], ] ] raw_ifmap_stats= [ [ _iter307->first = link_delete_parse_errors _iter307->second = 0, _iter307->first = link_update_parse_errors _iter307->second = 0, _iter307->first = node_delete_parse_errors _iter307->second = 0, _iter307->first = node_update_parse_errors _iter307->second = 0, ] ] raw_rt_table_size= [ [ _iter308->first = default-domain:default-project:ip-fabric:__default__ [ inet4_unicast = 4 inet4_multicast = 0 evpn = 0 bridge = 2 inet6_unicast = 2 ], _iter308->first = default-domain:default-project:ip-fabric:ip-fabric [ inet4_unicast = 2 inet4_multicast = 0 evpn = 1 bridge = 2 inet6_unicast = 0 ], ] ] ] 2025-09-01 Mon 02:09:25:759.319 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: VrouterControlStatsTrace 2025-09-01 Mon 02:09:26:807.993 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: Processing scm::EvIdleHoldTimerExpired in state Idle 2025-09-01 Mon 02:09:26:808.337 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: Connect : Start Connect timer 10.0.0.254:8086 2025-09-01 Mon 02:09:26:808.354 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848886536832, Pid 111888]: -(-1) OnSessionEvent TCP Connect Failed 2025-09-01 Mon 02:09:26:808.390 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848886536832, Pid 111888]: -(-1):EvTcpConnectFail 2025-09-01 Mon 02:09:26:808.474 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: Processing scm::EvSandeshSend in state Connect 2025-09-01 Mon 02:09:26:808.511 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: SANDESH: Send FAILED: 1756692566808121 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-3992-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.17 server_addrs= [ [ (*_iter6) = 10.20.0.17:5269, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.17 server_addrs= [ [ (*_iter6) = 10.20.0.17:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.254:8086, ] ] status = Initializing description = Idle : EvIdleHoldTimerExpired -> Connect ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.17, XMPP:dns-server:10.20.0.17, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:09:26:808.524 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: Wrong state: Connect for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:09:26:808.551 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: Processing scm::EvTcpConnectFail in state Connect 2025-09-01 Mon 02:09:26:808.561 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: Connect : EvTcpConnectFail 2025-09-01 Mon 02:09:26:808.617 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: TCP [SYS_ERR]: TcpSessionMessageLog: Session -(-1) > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:09:26:808.655 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: Idle 2025-09-01 Mon 02:09:26:808.738 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:26:808.952 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: SANDESH: Send FAILED: 1756692566808452 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3992-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Connect successful_connections = 0 pid = 111888 http_port = 8085 start_time = 1756692145743123 collector_name = collector_ip = 10.0.0.254:8086 collector_list= [ [ (*_iter40) = 10.0.0.254:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ComputeCpuStateTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 7 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 41 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 66 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 99 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterControlStatsTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 14 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 21 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 144 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 37 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 139, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 298, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:09:26:809.005 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:09:26:809.024 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:26:809.060 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: SANDESH: Send FAILED: 1756692566808589 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-3992-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.17 server_addrs= [ [ (*_iter6) = 10.20.0.17:5269, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.17 server_addrs= [ [ (*_iter6) = 10.20.0.17:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.17, XMPP:dns-server:10.20.0.17, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:09:26:809.076 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:09:26:809.088 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: Processing scm::EvTcpDeleteSession in state Idle 2025-09-01 Mon 02:09:26:809.123 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:26:809.399 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: SANDESH: Send FAILED: 1756692566808708 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3992-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Idle successful_connections = 0 pid = 111888 http_port = 8085 start_time = 1756692145743123 collector_name = collector_ip = 10.0.0.254:8086 collector_list= [ [ (*_iter40) = 10.0.0.254:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ComputeCpuStateTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 7 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 42 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 66 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 100 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterControlStatsTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 14 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 21 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 144 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 37 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 140, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 299, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:09:26:809.433 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:09:29:718.516 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848866952704, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:29:718.722 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848866952704, Pid 111888]: SANDESH: Send FAILED: 1756692569718454 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.17 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:29:718.776 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848866952704, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:29:718.935 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:29:718.939 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848886536832, Pid 111888]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.17 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:09:29:719.253 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848858555904, Pid 111888]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:09:29:719.057 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: SANDESH: Send FAILED: 1756692569718864 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.17 state = Connect event = Connect Event local_port = 33465 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:29:719.440 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:29:719.474 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:29:719.510 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: SANDESH: Send FAILED: 1756692569719415 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.17 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:29:719.529 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:29:719.578 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:29:719.605 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: SANDESH: Send FAILED: 1756692569719432 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.17 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:29:719.623 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:30:584.022 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848886536832, Pid 111888]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.20.0.17:8085::10.20.0.17:33692 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 255 2025-09-01 Mon 02:09:30:607.817 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848886536832, Pid 111888]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.20.0.17:8085::10.20.0.17:33704 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 255 2025-09-01 Mon 02:09:34:667.846 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848886536832, Pid 111888]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.0.0.254:8085::10.0.0.254:51566 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 255 2025-09-01 Mon 02:09:40:441.433 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:40:441.640 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: SANDESH: Send FAILED: 1756692580441327 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.17 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:40:441.726 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:40:441.754 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:40:441.780 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848886536832, Pid 111888]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.17 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:09:40:441.790 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: SANDESH: Send FAILED: 1756692580441527 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.17 state = Connect event = Connect Event local_port = 38303 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:40:441.934 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:40:441.986 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848858555904, Pid 111888]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:09:40:442.076 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848854357504, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:40:442.171 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848854357504, Pid 111888]: SANDESH: Send FAILED: 1756692580442034 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.17 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:40:442.199 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848854357504, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:40:442.224 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848854357504, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:40:442.279 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848854357504, Pid 111888]: SANDESH: Send FAILED: 1756692580442044 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.17 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:09:40:442.303 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848854357504, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:09:47:658.740 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848886536832, Pid 111888]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.0.0.254:8085::10.0.0.254:44514 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 255 2025-09-01 Mon 02:09:55:753.318 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848858555904, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:55:753.448 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848858555904, Pid 111888]: SANDESH: Send FAILED: 1756692595753087 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3992-1:Compute:contrail-vrouter-agent:0 sm_queue_count = 0 sm_stats= [ ev_stats= [ [ [ event = scm::EvCollectorUpdate enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvIdleHoldTimerExpired enqueues = 18 dequeues = 18 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvSandeshSend enqueues = 310 dequeues = 310 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvStart enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpConnectFail enqueues = 19 dequeues = 19 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpDeleteSession enqueues = 19 dequeues = 19 enqueue_fails = 0 dequeue_fails = 0 ], [ event = enqueues = 368 dequeues = 368 enqueue_fails = 0 dequeue_fails = 0 ], ] ] state = Idle state_since = 1756692566808653 last_state = Idle last_event = scm::EvSandeshSend last_event_at = 1756692580442220 ] max_sm_queue_count = 6 ] 2025-09-01 Mon 02:09:55:753.524 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848858555904, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:09:55:753.608 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848858555904, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:55:753.831 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848858555904, Pid 111888]: SANDESH: Send FAILED: 1756692595753257 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3992-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Idle successful_connections = 0 pid = 111888 http_port = 8085 start_time = 1756692145743123 collector_name = collector_ip = 10.0.0.254:8086 collector_list= [ [ (*_iter40) = 10.0.0.254:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ComputeCpuStateTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 7 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 43 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 68 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 106 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterControlStatsTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 14 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 21 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 152 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 39 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 148, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 310, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:09:55:753.863 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848858555904, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:09:55:758.416 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848854357504, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:55:758.605 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848854357504, Pid 111888]: SANDESH: Send FAILED: 1756692595758387 [SYS_NOTICE]: VrouterStats: data= [ name = cn-jenkins-deploy-platform-ansible-os-3992-1 in_tpkts = 171 in_bytes = 7350 out_tpkts = 190 out_bytes = 8836 raw_vhost_stats= [ name = vhost0 in_pkts = 171 in_bytes = 7350 out_pkts = 190 out_bytes = 8836 speed = -1 duplexity = -1 ] raw_vhost_drop_stats= [ ds_discard = 0 ds_pull = 0 ds_invalid_if = 0 ds_invalid_arp = 0 ds_trap_no_if = 0 ds_nowhere_to_go = 0 ds_flow_queue_limit_exceeded = 0 ds_flow_no_memory = 0 ds_flow_invalid_protocol = 0 ds_flow_nat_no_rflow = 0 ds_flow_action_drop = 0 ds_flow_action_invalid = 0 ds_flow_unusable = 0 ds_flow_table_full = 0 ds_interface_tx_discard = 0 ds_interface_drop = 0 ds_duplicated = 0 ds_push = 0 ds_ttl_exceeded = 0 ds_invalid_nh = 0 ds_invalid_label = 0 ds_invalid_protocol = 0 ds_interface_rx_discard = 0 ds_invalid_mcast_source = 0 ds_head_alloc_fail = 0 ds_pcow_fail = 0 ds_mcast_clone_fail = 0 ds_rewrite_fail = 0 ds_misc = 0 ds_invalid_packet = 0 ds_cksum_err = 0 ds_no_fmd = 0 ds_invalid_vnid = 0 ds_frag_err = 0 ds_invalid_source = 0 ds_mcast_df_bit = 0 ds_l2_no_route = 0 ds_vlan_fwd_tx = 0 ds_vlan_fwd_enq = 0 ds_drop_new_flow = 0 ds_flow_evict = 0 ds_trap_original = 0 ds_no_memory = 0 ds_fragment_queue_fail = 0 ds_no_frag_entry = 0 ds_icmp_error = 0 ds_clone_fail = 0 ds_drop_pkts = 0 ] raw_drop_stats= [ ds_discard = 0 ds_pull = 0 ds_invalid_if = 0 ds_invalid_arp = 0 ds_trap_no_if = 0 ds_nowhere_to_go = 0 ds_flow_queue_limit_exceeded = 0 ds_flow_no_memory = 0 ds_flow_invalid_protocol = 0 ds_flow_nat_no_rflow = 0 ds_flow_action_drop = 0 ds_flow_action_invalid = 0 ds_flow_unusable = 0 ds_flow_table_full = 0 ds_interface_tx_discard = 0 ds_interface_drop = 0 ds_duplicated = 0 ds_push = 0 ds_ttl_exceeded = 0 ds_invalid_nh = 0 ds_invalid_label = 0 ds_invalid_protocol = 0 ds_interface_rx_discard = 0 ds_invalid_mcast_source = 0 ds_head_alloc_fail = 0 ds_pcow_fail = 0 ds_mcast_clone_fail = 0 ds_rewrite_fail = 0 ds_misc = 0 ds_invalid_packet = 0 ds_cksum_err = 0 ds_no_fmd = 0 ds_invalid_vnid = 0 ds_frag_err = 0 ds_invalid_source = 0 ds_mcast_df_bit = 0 ds_l2_no_route = 0 ds_vlan_fwd_tx = 0 ds_vlan_fwd_enq = 0 ds_drop_new_flow = 0 ds_flow_evict = 0 ds_trap_original = 0 ds_no_memory = 0 ds_fragment_queue_fail = 0 ds_no_frag_entry = 0 ds_icmp_error = 0 ds_clone_fail = 0 ds_drop_pkts = 0 ] flow_rate= [ added_flows = 0 max_flow_adds_per_second = 0 min_flow_adds_per_second = 0 deleted_flows = 0 max_flow_deletes_per_second = 0 min_flow_deletes_per_second = 0 active_flows = 0 hold_flows = 0 ] phy_band_in_bps= [ [ _iter260->first = ens6 _iter260->second = 9, ] ] phy_band_out_bps= [ [ _iter261->first = ens6 _iter261->second = 33, ] ] raw_phy_if_stats= [ [ _iter262->first = ens6 [ in_pkts = 25 in_bytes = 1906 out_pkts = 171 out_bytes = 7350 ], ] ] raw_phy_if_drop_stats= [ [ _iter263->first = ens6 [ ds_discard = 0 ds_pull = 0 ds_invalid_if = 0 ds_invalid_arp = 0 ds_trap_no_if = 0 ds_nowhere_to_go = 0 ds_flow_queue_limit_exceeded = 0 ds_flow_no_memory = 0 ds_flow_invalid_protocol = 0 ds_flow_nat_no_rflow = 0 ds_flow_action_drop = 0 ds_flow_action_invalid = 0 ds_flow_unusable = 0 ds_flow_table_full = 0 ds_interface_tx_discard = 0 ds_interface_drop = 0 ds_duplicated = 0 ds_push = 0 ds_ttl_exceeded = 0 ds_invalid_nh = 0 ds_invalid_label = 0 ds_invalid_protocol = 0 ds_interface_rx_discard = 0 ds_invalid_mcast_source = 0 ds_head_alloc_fail = 0 ds_pcow_fail = 0 ds_mcast_clone_fail = 0 ds_rewrite_fail = 0 ds_misc = 0 ds_invalid_packet = 0 ds_cksum_err = 0 ds_no_fmd = 0 ds_invalid_vnid = 0 ds_frag_err = 0 ds_invalid_source = 0 ds_mcast_df_bit = 0 ds_l2_no_route = 0 ds_vlan_fwd_tx = 0 ds_vlan_fwd_enq = 0 ds_drop_new_flow = 0 ds_flow_evict = 0 ds_trap_original = 0 ds_no_memory = 0 ds_fragment_queue_fail = 0 ds_no_frag_entry = 0 ds_icmp_error = 0 ds_clone_fail = 0 ds_drop_pkts = 0 ], ] ] ] 2025-09-01 Mon 02:09:55:758.638 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848854357504, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: VrouterStats 2025-09-01 Mon 02:09:55:758.649 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848854357504, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:55:758.673 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848854357504, Pid 111888]: SANDESH: Send FAILED: 1756692595758429 [SYS_NOTICE]: VrouterControlStatsTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3992-1 raw_xmpp_stats= [ [ _iter306->first = 10.20.0.17 [ reconnects = 0 in_msgs = 0 out_msgs = 0 ], ] ] raw_ifmap_stats= [ [ _iter307->first = link_delete_parse_errors _iter307->second = 0, _iter307->first = link_update_parse_errors _iter307->second = 0, _iter307->first = node_delete_parse_errors _iter307->second = 0, _iter307->first = node_update_parse_errors _iter307->second = 0, ] ] raw_rt_table_size= [ [ _iter308->first = default-domain:default-project:ip-fabric:__default__ [ inet4_unicast = 4 inet4_multicast = 0 evpn = 0 bridge = 2 inet6_unicast = 2 ], _iter308->first = default-domain:default-project:ip-fabric:ip-fabric [ inet4_unicast = 2 inet4_multicast = 0 evpn = 1 bridge = 2 inet6_unicast = 0 ], ] ] ] 2025-09-01 Mon 02:09:55:758.681 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848854357504, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: VrouterControlStatsTrace 2025-09-01 Mon 02:09:56:808.792 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: Processing scm::EvIdleHoldTimerExpired in state Idle 2025-09-01 Mon 02:09:56:809.094 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: Connect : Start Connect timer 10.0.0.254:8086 2025-09-01 Mon 02:09:56:809.124 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848886536832, Pid 111888]: -(-1) OnSessionEvent TCP Connect Failed 2025-09-01 Mon 02:09:56:809.157 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848886536832, Pid 111888]: -(-1):EvTcpConnectFail 2025-09-01 Mon 02:09:56:809.195 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: Processing scm::EvSandeshSend in state Connect 2025-09-01 Mon 02:09:56:809.224 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: SANDESH: Send FAILED: 1756692596808896 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-3992-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.17 server_addrs= [ [ (*_iter6) = 10.20.0.17:5269, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.17 server_addrs= [ [ (*_iter6) = 10.20.0.17:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.254:8086, ] ] status = Initializing description = Idle : EvIdleHoldTimerExpired -> Connect ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.17, XMPP:dns-server:10.20.0.17, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:09:56:809.233 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: Wrong state: Connect for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:09:56:809.252 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: Processing scm::EvTcpConnectFail in state Connect 2025-09-01 Mon 02:09:56:809.259 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: Connect : EvTcpConnectFail 2025-09-01 Mon 02:09:56:809.300 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: TCP [SYS_ERR]: TcpSessionMessageLog: Session -(-1) > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:09:56:809.329 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: Idle 2025-09-01 Mon 02:09:56:809.380 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:56:809.516 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: SANDESH: Send FAILED: 1756692596809178 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3992-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Connect successful_connections = 0 pid = 111888 http_port = 8085 start_time = 1756692145743123 collector_name = collector_ip = 10.0.0.254:8086 collector_list= [ [ (*_iter40) = 10.0.0.254:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ComputeCpuStateTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 7 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 43 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 70 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 106 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterControlStatsTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 15 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 22 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 152 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 39 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 148, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 314, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:09:56:809.556 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:09:56:809.568 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:56:809.584 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: SANDESH: Send FAILED: 1756692596809280 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-3992-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.17 server_addrs= [ [ (*_iter6) = 10.20.0.17:5269, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.17 server_addrs= [ [ (*_iter6) = 10.20.0.17:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.17, XMPP:dns-server:10.20.0.17, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:09:56:809.590 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:09:56:809.597 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: Processing scm::EvTcpDeleteSession in state Idle 2025-09-01 Mon 02:09:56:809.623 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:09:56:809.809 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: SANDESH: Send FAILED: 1756692596809365 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3992-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Idle successful_connections = 0 pid = 111888 http_port = 8085 start_time = 1756692145743123 collector_name = collector_ip = 10.0.0.254:8086 collector_list= [ [ (*_iter40) = 10.0.0.254:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ComputeCpuStateTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 7 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 44 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 70 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 107 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterControlStatsTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 15 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 22 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 152 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 39 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 149, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 315, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:09:56:809.832 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:10:00:229.751 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848866952704, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:00:229.822 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848866952704, Pid 111888]: SANDESH: Send FAILED: 1756692600229717 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.17 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:00:229.836 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848866952704, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:00:229.984 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848862754304, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:00:229.989 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848886536832, Pid 111888]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.17 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:10:00:230.030 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848862754304, Pid 111888]: SANDESH: Send FAILED: 1756692600229954 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.17 state = Connect event = Connect Event local_port = 44505 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:00:230.082 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848862754304, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:00:230.110 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848858555904, Pid 111888]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:10:00:230.210 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:00:230.253 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: SANDESH: Send FAILED: 1756692600230148 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.17 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:00:230.266 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:00:230.279 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:00:230.295 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: SANDESH: Send FAILED: 1756692600230167 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.17 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:00:230.305 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:00:279.891 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848886536832, Pid 111888]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.0.0.254:8085::10.0.0.254:37624 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 255 2025-09-01 Mon 02:10:09:602.414 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848879547904, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:09:602.511 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848879547904, Pid 111888]: SANDESH: Send FAILED: 1756692609602350 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.17 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:09:602.525 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848879547904, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:09:602.804 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848854357504, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:09:602.843 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848886536832, Pid 111888]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.17 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:10:09:602.907 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848854357504, Pid 111888]: SANDESH: Send FAILED: 1756692609602736 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.17 state = Connect event = Connect Event local_port = 38177 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:09:603.078 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848854357504, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:09:603.159 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848854357504, Pid 111888]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:10:09:603.245 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848866952704, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:09:603.281 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848866952704, Pid 111888]: SANDESH: Send FAILED: 1756692609603223 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.17 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:09:603.290 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848866952704, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:09:603.299 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848866952704, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:09:603.309 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848866952704, Pid 111888]: SANDESH: Send FAILED: 1756692609603237 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.17 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:09:603.315 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848866952704, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:13:234.635 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848886536832, Pid 111888]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.0.0.254:8085::10.0.0.254:57318 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 255 2025-09-01 Mon 02:10:25:753.698 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848866952704, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:25:754.402 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848866952704, Pid 111888]: SANDESH: Send FAILED: 1756692625753439 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3992-1:Compute:contrail-vrouter-agent:0 sm_queue_count = 0 sm_stats= [ ev_stats= [ [ [ event = scm::EvCollectorUpdate enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvIdleHoldTimerExpired enqueues = 19 dequeues = 19 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvSandeshSend enqueues = 326 dequeues = 326 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvStart enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpConnectFail enqueues = 20 dequeues = 20 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpDeleteSession enqueues = 20 dequeues = 20 enqueue_fails = 0 dequeue_fails = 0 ], [ event = enqueues = 387 dequeues = 387 enqueue_fails = 0 dequeue_fails = 0 ], ] ] state = Idle state_since = 1756692596809328 last_state = Idle last_event = scm::EvSandeshSend last_event_at = 1756692609603298 ] max_sm_queue_count = 6 ] 2025-09-01 Mon 02:10:25:754.478 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848866952704, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:10:25:754.525 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848866952704, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:25:754.944 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848866952704, Pid 111888]: SANDESH: Send FAILED: 1756692625753592 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3992-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Idle successful_connections = 0 pid = 111888 http_port = 8085 start_time = 1756692145743123 collector_name = collector_ip = 10.0.0.254:8086 collector_list= [ [ (*_iter40) = 10.0.0.254:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ComputeCpuStateTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 7 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 45 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 72 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 111 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterControlStatsTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 15 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 22 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 160 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 41 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 155, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 326, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:10:25:754.991 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848866952704, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:10:25:758.723 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848879547904, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:25:758.869 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848879547904, Pid 111888]: SANDESH: Send FAILED: 1756692625758674 [SYS_NOTICE]: VrouterStats: data= [ name = cn-jenkins-deploy-platform-ansible-os-3992-1 cpu_info= [ num_cpu = 8 sys_mem_info= [ total = 32 used = 0 free = 0 buffers = 0 cached = 510398136 node_type = ] meminfo= [ virt = 32561 peakvirt = 0 res = 0 ] cpuload= [ one_min_avg = 0.1575 five_min_avg = 0.1975 fifteen_min_avg = 0.2175 ] cpu_share = 0.270833 ] ] 2025-09-01 Mon 02:10:25:758.897 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848879547904, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: VrouterStats 2025-09-01 Mon 02:10:25:758.925 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848879547904, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:25:758.971 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848879547904, Pid 111888]: SANDESH: Send FAILED: 1756692625758699 [SYS_NOTICE]: ComputeCpuStateTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3992-1 cpu_info= [ [ [ mem_virt = 32561 cpu_share = 0.270833 used_sys_mem = 3784569160 one_min_cpuload = 0.1575 mem_res = 0 ], ] ] ] 2025-09-01 Mon 02:10:25:758.993 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848879547904, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: ComputeCpuStateTrace 2025-09-01 Mon 02:10:25:759.020 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848879547904, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:25:759.424 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848879547904, Pid 111888]: SANDESH: Send FAILED: 1756692625758731 [SYS_NOTICE]: VrouterStats: data= [ name = cn-jenkins-deploy-platform-ansible-os-3992-1 in_tpkts = 174 in_bytes = 7476 out_tpkts = 193 out_bytes = 8962 raw_vhost_stats= [ name = vhost0 in_pkts = 174 in_bytes = 7476 out_pkts = 193 out_bytes = 8962 speed = -1 duplexity = -1 ] raw_vhost_drop_stats= [ ds_discard = 0 ds_pull = 0 ds_invalid_if = 0 ds_invalid_arp = 0 ds_trap_no_if = 0 ds_nowhere_to_go = 0 ds_flow_queue_limit_exceeded = 0 ds_flow_no_memory = 0 ds_flow_invalid_protocol = 0 ds_flow_nat_no_rflow = 0 ds_flow_action_drop = 0 ds_flow_action_invalid = 0 ds_flow_unusable = 0 ds_flow_table_full = 0 ds_interface_tx_discard = 0 ds_interface_drop = 0 ds_duplicated = 0 ds_push = 0 ds_ttl_exceeded = 0 ds_invalid_nh = 0 ds_invalid_label = 0 ds_invalid_protocol = 0 ds_interface_rx_discard = 0 ds_invalid_mcast_source = 0 ds_head_alloc_fail = 0 ds_pcow_fail = 0 ds_mcast_clone_fail = 0 ds_rewrite_fail = 0 ds_misc = 0 ds_invalid_packet = 0 ds_cksum_err = 0 ds_no_fmd = 0 ds_invalid_vnid = 0 ds_frag_err = 0 ds_invalid_source = 0 ds_mcast_df_bit = 0 ds_l2_no_route = 0 ds_vlan_fwd_tx = 0 ds_vlan_fwd_enq = 0 ds_drop_new_flow = 0 ds_flow_evict = 0 ds_trap_original = 0 ds_no_memory = 0 ds_fragment_queue_fail = 0 ds_no_frag_entry = 0 ds_icmp_error = 0 ds_clone_fail = 0 ds_drop_pkts = 0 ] raw_drop_stats= [ ds_discard = 0 ds_pull = 0 ds_invalid_if = 0 ds_invalid_arp = 0 ds_trap_no_if = 0 ds_nowhere_to_go = 0 ds_flow_queue_limit_exceeded = 0 ds_flow_no_memory = 0 ds_flow_invalid_protocol = 0 ds_flow_nat_no_rflow = 0 ds_flow_action_drop = 0 ds_flow_action_invalid = 0 ds_flow_unusable = 0 ds_flow_table_full = 0 ds_interface_tx_discard = 0 ds_interface_drop = 0 ds_duplicated = 0 ds_push = 0 ds_ttl_exceeded = 0 ds_invalid_nh = 0 ds_invalid_label = 0 ds_invalid_protocol = 0 ds_interface_rx_discard = 0 ds_invalid_mcast_source = 0 ds_head_alloc_fail = 0 ds_pcow_fail = 0 ds_mcast_clone_fail = 0 ds_rewrite_fail = 0 ds_misc = 0 ds_invalid_packet = 0 ds_cksum_err = 0 ds_no_fmd = 0 ds_invalid_vnid = 0 ds_frag_err = 0 ds_invalid_source = 0 ds_mcast_df_bit = 0 ds_l2_no_route = 0 ds_vlan_fwd_tx = 0 ds_vlan_fwd_enq = 0 ds_drop_new_flow = 0 ds_flow_evict = 0 ds_trap_original = 0 ds_no_memory = 0 ds_fragment_queue_fail = 0 ds_no_frag_entry = 0 ds_icmp_error = 0 ds_clone_fail = 0 ds_drop_pkts = 0 ] flow_rate= [ added_flows = 0 max_flow_adds_per_second = 0 min_flow_adds_per_second = 0 deleted_flows = 0 max_flow_deletes_per_second = 0 min_flow_deletes_per_second = 0 active_flows = 0 hold_flows = 0 ] raw_phy_if_stats= [ [ _iter262->first = ens6 [ in_pkts = 25 in_bytes = 1906 out_pkts = 174 out_bytes = 7476 ], ] ] raw_phy_if_drop_stats= [ [ _iter263->first = ens6 [ ds_discard = 0 ds_pull = 0 ds_invalid_if = 0 ds_invalid_arp = 0 ds_trap_no_if = 0 ds_nowhere_to_go = 0 ds_flow_queue_limit_exceeded = 0 ds_flow_no_memory = 0 ds_flow_invalid_protocol = 0 ds_flow_nat_no_rflow = 0 ds_flow_action_drop = 0 ds_flow_action_invalid = 0 ds_flow_unusable = 0 ds_flow_table_full = 0 ds_interface_tx_discard = 0 ds_interface_drop = 0 ds_duplicated = 0 ds_push = 0 ds_ttl_exceeded = 0 ds_invalid_nh = 0 ds_invalid_label = 0 ds_invalid_protocol = 0 ds_interface_rx_discard = 0 ds_invalid_mcast_source = 0 ds_head_alloc_fail = 0 ds_pcow_fail = 0 ds_mcast_clone_fail = 0 ds_rewrite_fail = 0 ds_misc = 0 ds_invalid_packet = 0 ds_cksum_err = 0 ds_no_fmd = 0 ds_invalid_vnid = 0 ds_frag_err = 0 ds_invalid_source = 0 ds_mcast_df_bit = 0 ds_l2_no_route = 0 ds_vlan_fwd_tx = 0 ds_vlan_fwd_enq = 0 ds_drop_new_flow = 0 ds_flow_evict = 0 ds_trap_original = 0 ds_no_memory = 0 ds_fragment_queue_fail = 0 ds_no_frag_entry = 0 ds_icmp_error = 0 ds_clone_fail = 0 ds_drop_pkts = 0 ], ] ] ] 2025-09-01 Mon 02:10:25:759.464 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848879547904, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: VrouterStats 2025-09-01 Mon 02:10:25:759.490 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848879547904, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:25:759.608 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848879547904, Pid 111888]: SANDESH: Send FAILED: 1756692625758763 [SYS_NOTICE]: VrouterControlStatsTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3992-1 raw_xmpp_stats= [ [ _iter306->first = 10.20.0.17 [ reconnects = 0 in_msgs = 0 out_msgs = 0 ], ] ] raw_ifmap_stats= [ [ _iter307->first = link_delete_parse_errors _iter307->second = 0, _iter307->first = link_update_parse_errors _iter307->second = 0, _iter307->first = node_delete_parse_errors _iter307->second = 0, _iter307->first = node_update_parse_errors _iter307->second = 0, ] ] raw_rt_table_size= [ [ _iter308->first = default-domain:default-project:ip-fabric:__default__ [ inet4_unicast = 4 inet4_multicast = 0 evpn = 0 bridge = 2 inet6_unicast = 2 ], _iter308->first = default-domain:default-project:ip-fabric:ip-fabric [ inet4_unicast = 2 inet4_multicast = 0 evpn = 1 bridge = 2 inet6_unicast = 0 ], ] ] ] 2025-09-01 Mon 02:10:25:759.631 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848879547904, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: VrouterControlStatsTrace 2025-09-01 Mon 02:10:25:999.855 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848886536832, Pid 111888]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.0.0.254:8085::10.0.0.254:48372 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 255 2025-09-01 Mon 02:10:26:809.477 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Processing scm::EvIdleHoldTimerExpired in state Idle 2025-09-01 Mon 02:10:26:809.850 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Connect : Start Connect timer 10.0.0.254:8086 2025-09-01 Mon 02:10:26:809.869 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848886536832, Pid 111888]: -(-1) OnSessionEvent TCP Connect Failed 2025-09-01 Mon 02:10:26:809.899 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848886536832, Pid 111888]: -(-1):EvTcpConnectFail 2025-09-01 Mon 02:10:26:810.023 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Processing scm::EvSandeshSend in state Connect 2025-09-01 Mon 02:10:26:810.055 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: SANDESH: Send FAILED: 1756692626809602 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-3992-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.17 server_addrs= [ [ (*_iter6) = 10.20.0.17:5269, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.17 server_addrs= [ [ (*_iter6) = 10.20.0.17:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.254:8086, ] ] status = Initializing description = Idle : EvIdleHoldTimerExpired -> Connect ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.17, XMPP:dns-server:10.20.0.17, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:10:26:810.066 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Wrong state: Connect for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:10:26:810.077 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Processing scm::EvTcpConnectFail in state Connect 2025-09-01 Mon 02:10:26:810.085 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Connect : EvTcpConnectFail 2025-09-01 Mon 02:10:26:810.147 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: TCP [SYS_ERR]: TcpSessionMessageLog: Session -(-1) > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:10:26:810.202 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Idle 2025-09-01 Mon 02:10:26:810.265 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:26:810.431 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: SANDESH: Send FAILED: 1756692626809989 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3992-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Connect successful_connections = 0 pid = 111888 http_port = 8085 start_time = 1756692145743123 collector_name = collector_ip = 10.0.0.254:8086 collector_list= [ [ (*_iter40) = 10.0.0.254:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ComputeCpuStateTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 8 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 45 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 74 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 112 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterControlStatsTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 16 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 24 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 160 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 41 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 156, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 332, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:10:26:810.454 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:10:26:810.470 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:26:810.489 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: SANDESH: Send FAILED: 1756692626810117 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-3992-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.17 server_addrs= [ [ (*_iter6) = 10.20.0.17:5269, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.17 server_addrs= [ [ (*_iter6) = 10.20.0.17:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.17, XMPP:dns-server:10.20.0.17, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:10:26:810.497 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:10:26:810.506 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Processing scm::EvTcpDeleteSession in state Idle 2025-09-01 Mon 02:10:26:810.553 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:26:810.699 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: SANDESH: Send FAILED: 1756692626810251 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3992-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Idle successful_connections = 0 pid = 111888 http_port = 8085 start_time = 1756692145743123 collector_name = collector_ip = 10.0.0.254:8086 collector_list= [ [ (*_iter40) = 10.0.0.254:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ComputeCpuStateTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 8 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 46 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 74 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 113 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterControlStatsTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 16 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 24 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 160 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 41 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 157, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 333, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:10:26:810.718 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:10:27:500.556 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848858555904, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:27:500.632 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848858555904, Pid 111888]: SANDESH: Send FAILED: 1756692627500489 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.17 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:27:500.660 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848858555904, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:27:500.955 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848862754304, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:27:500.975 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848862754304, Pid 111888]: SANDESH: Send FAILED: 1756692627500900 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.17 state = Connect event = Connect Event local_port = 43311 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:27:500.977 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848886536832, Pid 111888]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.17 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:10:27:500.984 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848862754304, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:27:501.170 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848858555904, Pid 111888]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:10:27:501.437 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848866952704, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:27:501.529 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848866952704, Pid 111888]: SANDESH: Send FAILED: 1756692627501294 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.17 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:27:501.581 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848866952704, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:27:501.614 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848866952704, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:27:501.707 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848866952704, Pid 111888]: SANDESH: Send FAILED: 1756692627501353 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.17 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:27:501.745 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848866952704, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:31:445.400 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848886536832, Pid 111888]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.20.0.17:8085::10.20.0.17:35366 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 255 2025-09-01 Mon 02:10:31:462.005 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848886536832, Pid 111888]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.20.0.17:8085::10.20.0.17:35380 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 255 2025-09-01 Mon 02:10:39:023.859 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848886536832, Pid 111888]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.0.0.254:8085::10.0.0.254:59222 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 255 2025-09-01 Mon 02:10:40:383.686 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848862754304, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:40:384.032 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848886536832, Pid 111888]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.17 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:10:40:384.477 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848862754304, Pid 111888]: SANDESH: Send FAILED: 1756692640383576 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.17 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:40:384.664 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848862754304, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:40:384.708 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848862754304, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:40:384.732 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:10:40:384.770 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848862754304, Pid 111888]: SANDESH: Send FAILED: 1756692640383921 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.17 state = Connect event = Connect Event local_port = 35001 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:40:384.875 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848862754304, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:40:384.900 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848862754304, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:40:384.931 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848862754304, Pid 111888]: SANDESH: Send FAILED: 1756692640384874 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.17 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:40:384.949 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848862754304, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:40:384.970 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848862754304, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:40:385.000 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848862754304, Pid 111888]: SANDESH: Send FAILED: 1756692640384887 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.17 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:40:385.018 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848862754304, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:51:959.158 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848886536832, Pid 111888]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.0.0.254:8085::10.0.0.254:57720 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 255 2025-09-01 Mon 02:10:55:754.016 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:55:754.116 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: SANDESH: Send FAILED: 1756692655753838 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3992-1:Compute:contrail-vrouter-agent:0 sm_queue_count = 0 sm_stats= [ ev_stats= [ [ [ event = scm::EvCollectorUpdate enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvIdleHoldTimerExpired enqueues = 20 dequeues = 20 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvSandeshSend enqueues = 344 dequeues = 344 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvStart enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpConnectFail enqueues = 21 dequeues = 21 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpDeleteSession enqueues = 21 dequeues = 21 enqueue_fails = 0 dequeue_fails = 0 ], [ event = enqueues = 408 dequeues = 408 enqueue_fails = 0 dequeue_fails = 0 ], ] ] state = Idle state_since = 1756692626810201 last_state = Idle last_event = scm::EvSandeshSend last_event_at = 1756692640384966 ] max_sm_queue_count = 6 ] 2025-09-01 Mon 02:10:55:754.148 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:10:55:754.167 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:55:754.405 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: SANDESH: Send FAILED: 1756692655753974 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3992-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Idle successful_connections = 0 pid = 111888 http_port = 8085 start_time = 1756692145743123 collector_name = collector_ip = 10.0.0.254:8086 collector_list= [ [ (*_iter40) = 10.0.0.254:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ComputeCpuStateTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 8 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 47 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 76 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 119 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterControlStatsTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 16 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 24 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 168 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 43 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 165, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 344, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:10:55:754.430 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:10:55:759.092 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848866952704, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:55:759.613 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848866952704, Pid 111888]: SANDESH: Send FAILED: 1756692655759036 [SYS_NOTICE]: VrouterStats: data= [ name = cn-jenkins-deploy-platform-ansible-os-3992-1 in_tpkts = 178 in_bytes = 7672 out_tpkts = 197 out_bytes = 9158 raw_vhost_stats= [ name = vhost0 in_pkts = 178 in_bytes = 7672 out_pkts = 197 out_bytes = 9158 speed = -1 duplexity = -1 ] raw_vhost_drop_stats= [ ds_discard = 0 ds_pull = 0 ds_invalid_if = 0 ds_invalid_arp = 0 ds_trap_no_if = 0 ds_nowhere_to_go = 0 ds_flow_queue_limit_exceeded = 0 ds_flow_no_memory = 0 ds_flow_invalid_protocol = 0 ds_flow_nat_no_rflow = 0 ds_flow_action_drop = 0 ds_flow_action_invalid = 0 ds_flow_unusable = 0 ds_flow_table_full = 0 ds_interface_tx_discard = 0 ds_interface_drop = 0 ds_duplicated = 0 ds_push = 0 ds_ttl_exceeded = 0 ds_invalid_nh = 0 ds_invalid_label = 0 ds_invalid_protocol = 0 ds_interface_rx_discard = 0 ds_invalid_mcast_source = 0 ds_head_alloc_fail = 0 ds_pcow_fail = 0 ds_mcast_clone_fail = 0 ds_rewrite_fail = 0 ds_misc = 0 ds_invalid_packet = 0 ds_cksum_err = 0 ds_no_fmd = 0 ds_invalid_vnid = 0 ds_frag_err = 0 ds_invalid_source = 0 ds_mcast_df_bit = 0 ds_l2_no_route = 0 ds_vlan_fwd_tx = 0 ds_vlan_fwd_enq = 0 ds_drop_new_flow = 0 ds_flow_evict = 0 ds_trap_original = 0 ds_no_memory = 0 ds_fragment_queue_fail = 0 ds_no_frag_entry = 0 ds_icmp_error = 0 ds_clone_fail = 0 ds_drop_pkts = 0 ] raw_drop_stats= [ ds_discard = 0 ds_pull = 0 ds_invalid_if = 0 ds_invalid_arp = 0 ds_trap_no_if = 0 ds_nowhere_to_go = 0 ds_flow_queue_limit_exceeded = 0 ds_flow_no_memory = 0 ds_flow_invalid_protocol = 0 ds_flow_nat_no_rflow = 0 ds_flow_action_drop = 0 ds_flow_action_invalid = 0 ds_flow_unusable = 0 ds_flow_table_full = 0 ds_interface_tx_discard = 0 ds_interface_drop = 0 ds_duplicated = 0 ds_push = 0 ds_ttl_exceeded = 0 ds_invalid_nh = 0 ds_invalid_label = 0 ds_invalid_protocol = 0 ds_interface_rx_discard = 0 ds_invalid_mcast_source = 0 ds_head_alloc_fail = 0 ds_pcow_fail = 0 ds_mcast_clone_fail = 0 ds_rewrite_fail = 0 ds_misc = 0 ds_invalid_packet = 0 ds_cksum_err = 0 ds_no_fmd = 0 ds_invalid_vnid = 0 ds_frag_err = 0 ds_invalid_source = 0 ds_mcast_df_bit = 0 ds_l2_no_route = 0 ds_vlan_fwd_tx = 0 ds_vlan_fwd_enq = 0 ds_drop_new_flow = 0 ds_flow_evict = 0 ds_trap_original = 0 ds_no_memory = 0 ds_fragment_queue_fail = 0 ds_no_frag_entry = 0 ds_icmp_error = 0 ds_clone_fail = 0 ds_drop_pkts = 0 ] total_out_bandwidth_utilization = 2.27682e-16 flow_rate= [ added_flows = 0 max_flow_adds_per_second = 0 min_flow_adds_per_second = 0 deleted_flows = 0 max_flow_deletes_per_second = 0 min_flow_deletes_per_second = 0 active_flows = 0 hold_flows = 0 ] phy_band_in_bps= [ [ _iter260->first = ens6 _iter260->second = 9, ] ] phy_band_out_bps= [ [ _iter261->first = ens6 _iter261->second = 42, ] ] raw_phy_if_stats= [ [ _iter262->first = ens6 [ in_pkts = 26 in_bytes = 1976 out_pkts = 178 out_bytes = 7672 ], ] ] raw_phy_if_drop_stats= [ [ _iter263->first = ens6 [ ds_discard = 0 ds_pull = 0 ds_invalid_if = 0 ds_invalid_arp = 0 ds_trap_no_if = 0 ds_nowhere_to_go = 0 ds_flow_queue_limit_exceeded = 0 ds_flow_no_memory = 0 ds_flow_invalid_protocol = 0 ds_flow_nat_no_rflow = 0 ds_flow_action_drop = 0 ds_flow_action_invalid = 0 ds_flow_unusable = 0 ds_flow_table_full = 0 ds_interface_tx_discard = 0 ds_interface_drop = 0 ds_duplicated = 0 ds_push = 0 ds_ttl_exceeded = 0 ds_invalid_nh = 0 ds_invalid_label = 0 ds_invalid_protocol = 0 ds_interface_rx_discard = 0 ds_invalid_mcast_source = 0 ds_head_alloc_fail = 0 ds_pcow_fail = 0 ds_mcast_clone_fail = 0 ds_rewrite_fail = 0 ds_misc = 0 ds_invalid_packet = 0 ds_cksum_err = 0 ds_no_fmd = 0 ds_invalid_vnid = 0 ds_frag_err = 0 ds_invalid_source = 0 ds_mcast_df_bit = 0 ds_l2_no_route = 0 ds_vlan_fwd_tx = 0 ds_vlan_fwd_enq = 0 ds_drop_new_flow = 0 ds_flow_evict = 0 ds_trap_original = 0 ds_no_memory = 0 ds_fragment_queue_fail = 0 ds_no_frag_entry = 0 ds_icmp_error = 0 ds_clone_fail = 0 ds_drop_pkts = 0 ], ] ] ] 2025-09-01 Mon 02:10:55:759.690 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848866952704, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: VrouterStats 2025-09-01 Mon 02:10:55:759.722 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848866952704, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:55:759.797 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848866952704, Pid 111888]: SANDESH: Send FAILED: 1756692655759103 [SYS_NOTICE]: VrouterControlStatsTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3992-1 raw_xmpp_stats= [ [ _iter306->first = 10.20.0.17 [ reconnects = 0 in_msgs = 0 out_msgs = 0 ], ] ] raw_ifmap_stats= [ [ _iter307->first = link_delete_parse_errors _iter307->second = 0, _iter307->first = link_update_parse_errors _iter307->second = 0, _iter307->first = node_delete_parse_errors _iter307->second = 0, _iter307->first = node_update_parse_errors _iter307->second = 0, ] ] raw_rt_table_size= [ [ _iter308->first = default-domain:default-project:ip-fabric:__default__ [ inet4_unicast = 4 inet4_multicast = 0 evpn = 0 bridge = 2 inet6_unicast = 2 ], _iter308->first = default-domain:default-project:ip-fabric:ip-fabric [ inet4_unicast = 2 inet4_multicast = 0 evpn = 1 bridge = 2 inet6_unicast = 0 ], ] ] ] 2025-09-01 Mon 02:10:55:759.817 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848866952704, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: VrouterControlStatsTrace 2025-09-01 Mon 02:10:56:810.465 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848866952704, Pid 111888]: Processing scm::EvIdleHoldTimerExpired in state Idle 2025-09-01 Mon 02:10:56:810.723 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848866952704, Pid 111888]: Connect : Start Connect timer 10.0.0.254:8086 2025-09-01 Mon 02:10:56:810.828 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848866952704, Pid 111888]: Processing scm::EvSandeshSend in state Connect 2025-09-01 Mon 02:10:56:810.849 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848886536832, Pid 111888]: -(-1) OnSessionEvent TCP Connect Failed 2025-09-01 Mon 02:10:56:810.852 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848866952704, Pid 111888]: SANDESH: Send FAILED: 1756692656810557 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-3992-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.17 server_addrs= [ [ (*_iter6) = 10.20.0.17:5269, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.17 server_addrs= [ [ (*_iter6) = 10.20.0.17:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.254:8086, ] ] status = Initializing description = Idle : EvIdleHoldTimerExpired -> Connect ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.17, XMPP:dns-server:10.20.0.17, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:10:56:810.944 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848866952704, Pid 111888]: Wrong state: Connect for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:10:56:810.953 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848866952704, Pid 111888]: Processing scm::EvSandeshSend in state Connect 2025-09-01 Mon 02:10:56:810.930 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848886536832, Pid 111888]: -(-1):EvTcpConnectFail 2025-09-01 Mon 02:10:56:811.067 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848866952704, Pid 111888]: SANDESH: Send FAILED: 1756692656810812 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3992-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Connect successful_connections = 0 pid = 111888 http_port = 8085 start_time = 1756692145743123 collector_name = collector_ip = 10.0.0.254:8086 collector_list= [ [ (*_iter40) = 10.0.0.254:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ComputeCpuStateTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 8 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 47 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 78 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 119 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterControlStatsTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 17 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 25 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 168 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 43 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 165, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 348, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:10:56:811.084 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848866952704, Pid 111888]: Wrong state: Connect for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:10:56:811.093 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848866952704, Pid 111888]: Processing scm::EvTcpConnectFail in state Connect 2025-09-01 Mon 02:10:56:811.099 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848866952704, Pid 111888]: Connect : EvTcpConnectFail 2025-09-01 Mon 02:10:56:811.135 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848866952704, Pid 111888]: TCP [SYS_ERR]: TcpSessionMessageLog: Session -(-1) > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:10:56:811.162 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848866952704, Pid 111888]: Idle 2025-09-01 Mon 02:10:56:811.205 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848866952704, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:56:811.220 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848866952704, Pid 111888]: SANDESH: Send FAILED: 1756692656811116 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-3992-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.17 server_addrs= [ [ (*_iter6) = 10.20.0.17:5269, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.17 server_addrs= [ [ (*_iter6) = 10.20.0.17:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.17, XMPP:dns-server:10.20.0.17, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:10:56:811.229 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848866952704, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:10:56:811.235 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848866952704, Pid 111888]: Processing scm::EvTcpDeleteSession in state Idle 2025-09-01 Mon 02:10:56:811.257 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848866952704, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:56:811.364 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848866952704, Pid 111888]: SANDESH: Send FAILED: 1756692656811198 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3992-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Idle successful_connections = 0 pid = 111888 http_port = 8085 start_time = 1756692145743123 collector_name = collector_ip = 10.0.0.254:8086 collector_list= [ [ (*_iter40) = 10.0.0.254:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ComputeCpuStateTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 8 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 48 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 79 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 120 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterControlStatsTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 17 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 25 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 168 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 43 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 166, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 350, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:10:56:811.381 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848866952704, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:10:57:741.850 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848879547904, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:57:741.959 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848879547904, Pid 111888]: SANDESH: Send FAILED: 1756692657741777 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.17 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:57:741.986 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848879547904, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:57:742.012 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848879547904, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:57:742.048 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848879547904, Pid 111888]: SANDESH: Send FAILED: 1756692657741985 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.17 state = Connect event = Connect Event local_port = 38873 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:57:742.067 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848879547904, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:57:742.076 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848886536832, Pid 111888]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.17 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:10:57:742.226 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:10:57:742.288 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848879547904, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:57:742.323 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848879547904, Pid 111888]: SANDESH: Send FAILED: 1756692657742274 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.17 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:57:742.341 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848879547904, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:10:57:742.362 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848879547904, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:10:57:742.388 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848879547904, Pid 111888]: SANDESH: Send FAILED: 1756692657742280 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.17 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:10:57:742.405 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848879547904, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:04:767.064 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848886536832, Pid 111888]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.0.0.254:8085::10.0.0.254:44384 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 255 2025-09-01 Mon 02:11:11:165.286 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848862754304, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:11:165.370 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848862754304, Pid 111888]: SANDESH: Send FAILED: 1756692671165248 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.17 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:11:165.385 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848862754304, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:11:165.574 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848866952704, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:11:165.630 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848866952704, Pid 111888]: SANDESH: Send FAILED: 1756692671165525 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.17 state = Connect event = Connect Event local_port = 37077 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:11:165.670 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848866952704, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:11:165.713 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848886536832, Pid 111888]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.17 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:11:11:165.964 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:11:11:166.197 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848858555904, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:11:166.237 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848858555904, Pid 111888]: SANDESH: Send FAILED: 1756692671166070 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.17 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:11:166.262 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848858555904, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:11:166.271 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848858555904, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:11:166.282 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848858555904, Pid 111888]: SANDESH: Send FAILED: 1756692671166126 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.17 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:11:166.303 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848858555904, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:17:974.945 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848886536832, Pid 111888]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.0.0.254:8085::10.0.0.254:34994 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 255 2025-09-01 Mon 02:11:25:754.599 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848854357504, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:25:754.866 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848854357504, Pid 111888]: SANDESH: Send FAILED: 1756692685754218 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3992-1:Compute:contrail-vrouter-agent:0 sm_queue_count = 0 sm_stats= [ ev_stats= [ [ [ event = scm::EvCollectorUpdate enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvIdleHoldTimerExpired enqueues = 21 dequeues = 21 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvSandeshSend enqueues = 360 dequeues = 360 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvStart enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpConnectFail enqueues = 22 dequeues = 22 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpDeleteSession enqueues = 22 dequeues = 22 enqueue_fails = 0 dequeue_fails = 0 ], [ event = enqueues = 427 dequeues = 427 enqueue_fails = 0 dequeue_fails = 0 ], ] ] state = Idle state_since = 1756692656811161 last_state = Idle last_event = scm::EvSandeshSend last_event_at = 1756692671166270 ] max_sm_queue_count = 6 ] 2025-09-01 Mon 02:11:25:754.911 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848854357504, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:11:25:754.943 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848854357504, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:25:755.599 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848854357504, Pid 111888]: SANDESH: Send FAILED: 1756692685754461 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3992-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Idle successful_connections = 0 pid = 111888 http_port = 8085 start_time = 1756692145743123 collector_name = collector_ip = 10.0.0.254:8086 collector_list= [ [ (*_iter40) = 10.0.0.254:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ComputeCpuStateTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 8 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 49 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 80 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 124 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterControlStatsTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 17 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 25 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 176 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 45 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 172, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 360, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:11:25:755.672 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848854357504, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:11:25:759.433 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848866952704, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:25:759.530 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848866952704, Pid 111888]: SANDESH: Send FAILED: 1756692685759358 [SYS_NOTICE]: VrouterStats: data= [ name = cn-jenkins-deploy-platform-ansible-os-3992-1 cpu_info= [ num_cpu = 8 sys_mem_info= [ total = 32 used = 0 free = 0 buffers = 0 cached = 510398136 node_type = ] meminfo= [ virt = 32561 peakvirt = 0 res = 0 ] cpuload= [ one_min_avg = 0.2125 five_min_avg = 0.205 fifteen_min_avg = 0.21875 ] cpu_share = 0.2875 ] ] 2025-09-01 Mon 02:11:25:759.549 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848866952704, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: VrouterStats 2025-09-01 Mon 02:11:25:759.559 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848866952704, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:25:759.573 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848866952704, Pid 111888]: SANDESH: Send FAILED: 1756692685759415 [SYS_NOTICE]: ComputeCpuStateTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3992-1 cpu_info= [ [ [ mem_virt = 32561 cpu_share = 0.2875 used_sys_mem = 3784569160 one_min_cpuload = 0.2125 mem_res = 0 ], ] ] ] 2025-09-01 Mon 02:11:25:759.579 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848866952704, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: ComputeCpuStateTrace 2025-09-01 Mon 02:11:25:759.588 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848866952704, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:25:759.690 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848866952704, Pid 111888]: SANDESH: Send FAILED: 1756692685759464 [SYS_NOTICE]: VrouterStats: data= [ name = cn-jenkins-deploy-platform-ansible-os-3992-1 in_tpkts = 181 in_bytes = 7798 out_tpkts = 200 out_bytes = 9284 exception_packets = 13 exception_packets_dropped = 13 raw_vhost_stats= [ name = vhost0 in_pkts = 181 in_bytes = 7798 out_pkts = 200 out_bytes = 9284 speed = -1 duplexity = -1 ] raw_vhost_drop_stats= [ ds_discard = 0 ds_pull = 0 ds_invalid_if = 0 ds_invalid_arp = 0 ds_trap_no_if = 0 ds_nowhere_to_go = 0 ds_flow_queue_limit_exceeded = 0 ds_flow_no_memory = 0 ds_flow_invalid_protocol = 0 ds_flow_nat_no_rflow = 0 ds_flow_action_drop = 0 ds_flow_action_invalid = 0 ds_flow_unusable = 0 ds_flow_table_full = 0 ds_interface_tx_discard = 0 ds_interface_drop = 0 ds_duplicated = 0 ds_push = 0 ds_ttl_exceeded = 0 ds_invalid_nh = 0 ds_invalid_label = 0 ds_invalid_protocol = 0 ds_interface_rx_discard = 0 ds_invalid_mcast_source = 0 ds_head_alloc_fail = 0 ds_pcow_fail = 0 ds_mcast_clone_fail = 0 ds_rewrite_fail = 0 ds_misc = 0 ds_invalid_packet = 0 ds_cksum_err = 0 ds_no_fmd = 0 ds_invalid_vnid = 0 ds_frag_err = 0 ds_invalid_source = 0 ds_mcast_df_bit = 0 ds_l2_no_route = 0 ds_vlan_fwd_tx = 0 ds_vlan_fwd_enq = 0 ds_drop_new_flow = 0 ds_flow_evict = 0 ds_trap_original = 0 ds_no_memory = 0 ds_fragment_queue_fail = 0 ds_no_frag_entry = 0 ds_icmp_error = 0 ds_clone_fail = 0 ds_drop_pkts = 0 ] raw_drop_stats= [ ds_discard = 0 ds_pull = 0 ds_invalid_if = 0 ds_invalid_arp = 0 ds_trap_no_if = 0 ds_nowhere_to_go = 0 ds_flow_queue_limit_exceeded = 0 ds_flow_no_memory = 0 ds_flow_invalid_protocol = 0 ds_flow_nat_no_rflow = 0 ds_flow_action_drop = 0 ds_flow_action_invalid = 0 ds_flow_unusable = 0 ds_flow_table_full = 0 ds_interface_tx_discard = 0 ds_interface_drop = 0 ds_duplicated = 0 ds_push = 0 ds_ttl_exceeded = 0 ds_invalid_nh = 0 ds_invalid_label = 0 ds_invalid_protocol = 0 ds_interface_rx_discard = 0 ds_invalid_mcast_source = 0 ds_head_alloc_fail = 0 ds_pcow_fail = 0 ds_mcast_clone_fail = 0 ds_rewrite_fail = 0 ds_misc = 0 ds_invalid_packet = 0 ds_cksum_err = 0 ds_no_fmd = 0 ds_invalid_vnid = 0 ds_frag_err = 0 ds_invalid_source = 0 ds_mcast_df_bit = 0 ds_l2_no_route = 0 ds_vlan_fwd_tx = 0 ds_vlan_fwd_enq = 0 ds_drop_new_flow = 0 ds_flow_evict = 0 ds_trap_original = 0 ds_no_memory = 0 ds_fragment_queue_fail = 0 ds_no_frag_entry = 0 ds_icmp_error = 0 ds_clone_fail = 0 ds_drop_pkts = 0 ] flow_rate= [ added_flows = 0 max_flow_adds_per_second = 0 min_flow_adds_per_second = 0 deleted_flows = 0 max_flow_deletes_per_second = 0 min_flow_deletes_per_second = 0 active_flows = 0 hold_flows = 0 ] raw_phy_if_stats= [ [ _iter262->first = ens6 [ in_pkts = 26 in_bytes = 1976 out_pkts = 181 out_bytes = 7798 ], ] ] raw_phy_if_drop_stats= [ [ _iter263->first = ens6 [ ds_discard = 0 ds_pull = 0 ds_invalid_if = 0 ds_invalid_arp = 0 ds_trap_no_if = 0 ds_nowhere_to_go = 0 ds_flow_queue_limit_exceeded = 0 ds_flow_no_memory = 0 ds_flow_invalid_protocol = 0 ds_flow_nat_no_rflow = 0 ds_flow_action_drop = 0 ds_flow_action_invalid = 0 ds_flow_unusable = 0 ds_flow_table_full = 0 ds_interface_tx_discard = 0 ds_interface_drop = 0 ds_duplicated = 0 ds_push = 0 ds_ttl_exceeded = 0 ds_invalid_nh = 0 ds_invalid_label = 0 ds_invalid_protocol = 0 ds_interface_rx_discard = 0 ds_invalid_mcast_source = 0 ds_head_alloc_fail = 0 ds_pcow_fail = 0 ds_mcast_clone_fail = 0 ds_rewrite_fail = 0 ds_misc = 0 ds_invalid_packet = 0 ds_cksum_err = 0 ds_no_fmd = 0 ds_invalid_vnid = 0 ds_frag_err = 0 ds_invalid_source = 0 ds_mcast_df_bit = 0 ds_l2_no_route = 0 ds_vlan_fwd_tx = 0 ds_vlan_fwd_enq = 0 ds_drop_new_flow = 0 ds_flow_evict = 0 ds_trap_original = 0 ds_no_memory = 0 ds_fragment_queue_fail = 0 ds_no_frag_entry = 0 ds_icmp_error = 0 ds_clone_fail = 0 ds_drop_pkts = 0 ], ] ] ] 2025-09-01 Mon 02:11:25:759.705 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848866952704, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: VrouterStats 2025-09-01 Mon 02:11:25:759.712 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848866952704, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:25:759.734 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848866952704, Pid 111888]: SANDESH: Send FAILED: 1756692685759535 [SYS_NOTICE]: VrouterControlStatsTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3992-1 raw_xmpp_stats= [ [ _iter306->first = 10.20.0.17 [ reconnects = 0 in_msgs = 0 out_msgs = 0 ], ] ] raw_ifmap_stats= [ [ _iter307->first = link_delete_parse_errors _iter307->second = 0, _iter307->first = link_update_parse_errors _iter307->second = 0, _iter307->first = node_delete_parse_errors _iter307->second = 0, _iter307->first = node_update_parse_errors _iter307->second = 0, ] ] raw_rt_table_size= [ [ _iter308->first = default-domain:default-project:ip-fabric:__default__ [ inet4_unicast = 4 inet4_multicast = 0 evpn = 0 bridge = 2 inet6_unicast = 2 ], _iter308->first = default-domain:default-project:ip-fabric:ip-fabric [ inet4_unicast = 2 inet4_multicast = 0 evpn = 1 bridge = 2 inet6_unicast = 0 ], ] ] ] 2025-09-01 Mon 02:11:25:759.740 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848866952704, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: VrouterControlStatsTrace 2025-09-01 Mon 02:11:26:811.495 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848879547904, Pid 111888]: Processing scm::EvIdleHoldTimerExpired in state Idle 2025-09-01 Mon 02:11:26:812.078 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848879547904, Pid 111888]: Connect : Start Connect timer 10.0.0.254:8086 2025-09-01 Mon 02:11:26:812.142 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848886536832, Pid 111888]: -(-1) OnSessionEvent TCP Connect Failed 2025-09-01 Mon 02:11:26:812.222 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848886536832, Pid 111888]: -(-1):EvTcpConnectFail 2025-09-01 Mon 02:11:26:812.326 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848879547904, Pid 111888]: Processing scm::EvSandeshSend in state Connect 2025-09-01 Mon 02:11:26:812.411 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848879547904, Pid 111888]: SANDESH: Send FAILED: 1756692686811715 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-3992-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.17 server_addrs= [ [ (*_iter6) = 10.20.0.17:5269, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.17 server_addrs= [ [ (*_iter6) = 10.20.0.17:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.254:8086, ] ] status = Initializing description = Idle : EvIdleHoldTimerExpired -> Connect ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.17, XMPP:dns-server:10.20.0.17, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:11:26:812.466 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848879547904, Pid 111888]: Wrong state: Connect for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:11:26:812.493 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848879547904, Pid 111888]: Processing scm::EvTcpConnectFail in state Connect 2025-09-01 Mon 02:11:26:812.514 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848879547904, Pid 111888]: Connect : EvTcpConnectFail 2025-09-01 Mon 02:11:26:812.640 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848879547904, Pid 111888]: TCP [SYS_ERR]: TcpSessionMessageLog: Session -(-1) > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:11:26:812.726 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848879547904, Pid 111888]: Idle 2025-09-01 Mon 02:11:26:812.876 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848879547904, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:26:813.272 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848879547904, Pid 111888]: SANDESH: Send FAILED: 1756692686812282 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3992-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Connect successful_connections = 0 pid = 111888 http_port = 8085 start_time = 1756692145743123 collector_name = collector_ip = 10.0.0.254:8086 collector_list= [ [ (*_iter40) = 10.0.0.254:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ComputeCpuStateTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 9 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 49 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 82 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 124 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterControlStatsTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 18 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 27 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 176 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 45 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 172, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 366, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:11:26:813.327 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848879547904, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:11:26:813.357 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848879547904, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:26:813.406 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848879547904, Pid 111888]: SANDESH: Send FAILED: 1756692686812589 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-3992-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.17 server_addrs= [ [ (*_iter6) = 10.20.0.17:5269, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.17 server_addrs= [ [ (*_iter6) = 10.20.0.17:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.17, XMPP:dns-server:10.20.0.17, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:11:26:813.425 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848879547904, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:11:26:813.448 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848879547904, Pid 111888]: Processing scm::EvTcpDeleteSession in state Idle 2025-09-01 Mon 02:11:26:813.519 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848879547904, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:26:814.013 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848879547904, Pid 111888]: SANDESH: Send FAILED: 1756692686812846 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3992-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Idle successful_connections = 0 pid = 111888 http_port = 8085 start_time = 1756692145743123 collector_name = collector_ip = 10.0.0.254:8086 collector_list= [ [ (*_iter40) = 10.0.0.254:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ComputeCpuStateTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 9 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 50 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 82 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 125 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterControlStatsTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 18 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 27 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 176 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 45 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 173, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 367, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:11:26:814.073 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848879547904, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:11:27:982.589 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848854357504, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:27:982.708 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848854357504, Pid 111888]: SANDESH: Send FAILED: 1756692687982523 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.17 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:27:982.746 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848854357504, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:27:982.782 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848886536832, Pid 111888]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.17 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:11:27:982.784 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848854357504, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:27:982.916 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848854357504, Pid 111888]: SANDESH: Send FAILED: 1756692687982681 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.17 state = Connect event = Connect Event local_port = 41039 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:27:982.928 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848879547904, Pid 111888]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:11:27:982.950 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848854357504, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:27:983.126 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:27:983.152 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: SANDESH: Send FAILED: 1756692687983012 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.17 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:27:983.161 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:27:983.172 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:27:983.192 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: SANDESH: Send FAILED: 1756692687983131 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.17 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:27:983.199 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:30:897.887 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848886536832, Pid 111888]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.0.0.254:8085::10.0.0.254:37258 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 255 2025-09-01 Mon 02:11:32:507.188 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848886536832, Pid 111888]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.20.0.17:8085::10.20.0.17:44516 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 255 2025-09-01 Mon 02:11:32:529.016 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848886536832, Pid 111888]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.20.0.17:8085::10.20.0.17:44528 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 255 2025-09-01 Mon 02:11:41:676.466 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848879547904, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:41:676.561 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848879547904, Pid 111888]: SANDESH: Send FAILED: 1756692701676422 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.17 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:41:676.576 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848879547904, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:41:676.970 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848886536832, Pid 111888]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.17 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:11:41:676.992 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848858555904, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:41:677.118 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848854357504, Pid 111888]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:11:41:677.198 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848858555904, Pid 111888]: SANDESH: Send FAILED: 1756692701676912 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.17 state = Connect event = Connect Event local_port = 34781 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:41:677.229 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848858555904, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:41:677.255 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848858555904, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:41:677.306 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848858555904, Pid 111888]: SANDESH: Send FAILED: 1756692701677216 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.17 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:41:677.326 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848858555904, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:41:677.370 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848858555904, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:41:677.398 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848858555904, Pid 111888]: SANDESH: Send FAILED: 1756692701677224 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.17 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:41:677.415 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848858555904, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:43:788.757 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848886536832, Pid 111888]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.0.0.254:8085::10.0.0.254:34118 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 255 2025-09-01 Mon 02:11:55:755.110 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848858555904, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:55:755.289 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848858555904, Pid 111888]: SANDESH: Send FAILED: 1756692715754790 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3992-1:Compute:contrail-vrouter-agent:0 sm_queue_count = 0 sm_stats= [ ev_stats= [ [ [ event = scm::EvCollectorUpdate enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvIdleHoldTimerExpired enqueues = 22 dequeues = 22 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvSandeshSend enqueues = 378 dequeues = 378 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvStart enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpConnectFail enqueues = 23 dequeues = 23 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpDeleteSession enqueues = 23 dequeues = 23 enqueue_fails = 0 dequeue_fails = 0 ], [ event = enqueues = 448 dequeues = 448 enqueue_fails = 0 dequeue_fails = 0 ], ] ] state = Idle state_since = 1756692686812723 last_state = Idle last_event = scm::EvSandeshSend last_event_at = 1756692701677366 ] max_sm_queue_count = 6 ] 2025-09-01 Mon 02:11:55:755.327 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848858555904, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:11:55:755.352 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848858555904, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:55:755.775 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848858555904, Pid 111888]: SANDESH: Send FAILED: 1756692715755006 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3992-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Idle successful_connections = 0 pid = 111888 http_port = 8085 start_time = 1756692145743123 collector_name = collector_ip = 10.0.0.254:8086 collector_list= [ [ (*_iter40) = 10.0.0.254:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ComputeCpuStateTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 9 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 51 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 84 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 131 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterControlStatsTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 18 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 27 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 184 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 47 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 181, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 378, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:11:55:755.838 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848858555904, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:11:55:759.916 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848858555904, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:55:760.289 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848858555904, Pid 111888]: SANDESH: Send FAILED: 1756692715759835 [SYS_NOTICE]: VrouterStats: data= [ name = cn-jenkins-deploy-platform-ansible-os-3992-1 in_tpkts = 184 in_bytes = 7924 out_tpkts = 203 out_bytes = 9410 raw_vhost_stats= [ name = vhost0 in_pkts = 184 in_bytes = 7924 out_pkts = 203 out_bytes = 9410 speed = -1 duplexity = -1 ] raw_vhost_drop_stats= [ ds_discard = 0 ds_pull = 0 ds_invalid_if = 0 ds_invalid_arp = 0 ds_trap_no_if = 0 ds_nowhere_to_go = 0 ds_flow_queue_limit_exceeded = 0 ds_flow_no_memory = 0 ds_flow_invalid_protocol = 0 ds_flow_nat_no_rflow = 0 ds_flow_action_drop = 0 ds_flow_action_invalid = 0 ds_flow_unusable = 0 ds_flow_table_full = 0 ds_interface_tx_discard = 0 ds_interface_drop = 0 ds_duplicated = 0 ds_push = 0 ds_ttl_exceeded = 0 ds_invalid_nh = 0 ds_invalid_label = 0 ds_invalid_protocol = 0 ds_interface_rx_discard = 0 ds_invalid_mcast_source = 0 ds_head_alloc_fail = 0 ds_pcow_fail = 0 ds_mcast_clone_fail = 0 ds_rewrite_fail = 0 ds_misc = 0 ds_invalid_packet = 0 ds_cksum_err = 0 ds_no_fmd = 0 ds_invalid_vnid = 0 ds_frag_err = 0 ds_invalid_source = 0 ds_mcast_df_bit = 0 ds_l2_no_route = 0 ds_vlan_fwd_tx = 0 ds_vlan_fwd_enq = 0 ds_drop_new_flow = 0 ds_flow_evict = 0 ds_trap_original = 0 ds_no_memory = 0 ds_fragment_queue_fail = 0 ds_no_frag_entry = 0 ds_icmp_error = 0 ds_clone_fail = 0 ds_drop_pkts = 0 ] raw_drop_stats= [ ds_discard = 0 ds_pull = 0 ds_invalid_if = 0 ds_invalid_arp = 0 ds_trap_no_if = 0 ds_nowhere_to_go = 0 ds_flow_queue_limit_exceeded = 0 ds_flow_no_memory = 0 ds_flow_invalid_protocol = 0 ds_flow_nat_no_rflow = 0 ds_flow_action_drop = 0 ds_flow_action_invalid = 0 ds_flow_unusable = 0 ds_flow_table_full = 0 ds_interface_tx_discard = 0 ds_interface_drop = 0 ds_duplicated = 0 ds_push = 0 ds_ttl_exceeded = 0 ds_invalid_nh = 0 ds_invalid_label = 0 ds_invalid_protocol = 0 ds_interface_rx_discard = 0 ds_invalid_mcast_source = 0 ds_head_alloc_fail = 0 ds_pcow_fail = 0 ds_mcast_clone_fail = 0 ds_rewrite_fail = 0 ds_misc = 0 ds_invalid_packet = 0 ds_cksum_err = 0 ds_no_fmd = 0 ds_invalid_vnid = 0 ds_frag_err = 0 ds_invalid_source = 0 ds_mcast_df_bit = 0 ds_l2_no_route = 0 ds_vlan_fwd_tx = 0 ds_vlan_fwd_enq = 0 ds_drop_new_flow = 0 ds_flow_evict = 0 ds_trap_original = 0 ds_no_memory = 0 ds_fragment_queue_fail = 0 ds_no_frag_entry = 0 ds_icmp_error = 0 ds_clone_fail = 0 ds_drop_pkts = 0 ] total_in_bandwidth_utilization = 0 total_out_bandwidth_utilization = 1.78893e-16 flow_rate= [ added_flows = 0 max_flow_adds_per_second = 0 min_flow_adds_per_second = 0 deleted_flows = 0 max_flow_deletes_per_second = 0 min_flow_deletes_per_second = 0 active_flows = 0 hold_flows = 0 ] phy_band_in_bps= [ [ _iter260->first = ens6 _iter260->second = 0, ] ] phy_band_out_bps= [ [ _iter261->first = ens6 _iter261->second = 33, ] ] raw_phy_if_stats= [ [ _iter262->first = ens6 [ in_pkts = 26 in_bytes = 1976 out_pkts = 184 out_bytes = 7924 ], ] ] raw_phy_if_drop_stats= [ [ _iter263->first = ens6 [ ds_discard = 0 ds_pull = 0 ds_invalid_if = 0 ds_invalid_arp = 0 ds_trap_no_if = 0 ds_nowhere_to_go = 0 ds_flow_queue_limit_exceeded = 0 ds_flow_no_memory = 0 ds_flow_invalid_protocol = 0 ds_flow_nat_no_rflow = 0 ds_flow_action_drop = 0 ds_flow_action_invalid = 0 ds_flow_unusable = 0 ds_flow_table_full = 0 ds_interface_tx_discard = 0 ds_interface_drop = 0 ds_duplicated = 0 ds_push = 0 ds_ttl_exceeded = 0 ds_invalid_nh = 0 ds_invalid_label = 0 ds_invalid_protocol = 0 ds_interface_rx_discard = 0 ds_invalid_mcast_source = 0 ds_head_alloc_fail = 0 ds_pcow_fail = 0 ds_mcast_clone_fail = 0 ds_rewrite_fail = 0 ds_misc = 0 ds_invalid_packet = 0 ds_cksum_err = 0 ds_no_fmd = 0 ds_invalid_vnid = 0 ds_frag_err = 0 ds_invalid_source = 0 ds_mcast_df_bit = 0 ds_l2_no_route = 0 ds_vlan_fwd_tx = 0 ds_vlan_fwd_enq = 0 ds_drop_new_flow = 0 ds_flow_evict = 0 ds_trap_original = 0 ds_no_memory = 0 ds_fragment_queue_fail = 0 ds_no_frag_entry = 0 ds_icmp_error = 0 ds_clone_fail = 0 ds_drop_pkts = 0 ], ] ] ] 2025-09-01 Mon 02:11:55:760.340 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848858555904, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: VrouterStats 2025-09-01 Mon 02:11:55:760.365 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848858555904, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:55:760.447 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848858555904, Pid 111888]: SANDESH: Send FAILED: 1756692715759938 [SYS_NOTICE]: VrouterControlStatsTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3992-1 raw_xmpp_stats= [ [ _iter306->first = 10.20.0.17 [ reconnects = 0 in_msgs = 0 out_msgs = 0 ], ] ] raw_ifmap_stats= [ [ _iter307->first = link_delete_parse_errors _iter307->second = 0, _iter307->first = link_update_parse_errors _iter307->second = 0, _iter307->first = node_delete_parse_errors _iter307->second = 0, _iter307->first = node_update_parse_errors _iter307->second = 0, ] ] raw_rt_table_size= [ [ _iter308->first = default-domain:default-project:ip-fabric:__default__ [ inet4_unicast = 4 inet4_multicast = 0 evpn = 0 bridge = 2 inet6_unicast = 2 ], _iter308->first = default-domain:default-project:ip-fabric:ip-fabric [ inet4_unicast = 2 inet4_multicast = 0 evpn = 1 bridge = 2 inet6_unicast = 0 ], ] ] ] 2025-09-01 Mon 02:11:55:760.467 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848858555904, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: VrouterControlStatsTrace 2025-09-01 Mon 02:11:56:766.479 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848886536832, Pid 111888]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.0.0.254:8085::10.0.0.254:58574 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 255 2025-09-01 Mon 02:11:56:812.948 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Processing scm::EvIdleHoldTimerExpired in state Idle 2025-09-01 Mon 02:11:56:813.378 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Connect : Start Connect timer 10.0.0.254:8086 2025-09-01 Mon 02:11:56:813.436 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848886536832, Pid 111888]: -(-1) OnSessionEvent TCP Connect Failed 2025-09-01 Mon 02:11:56:813.507 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848886536832, Pid 111888]: -(-1):EvTcpConnectFail 2025-09-01 Mon 02:11:56:813.613 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Processing scm::EvSandeshSend in state Connect 2025-09-01 Mon 02:11:56:813.732 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: SANDESH: Send FAILED: 1756692716813079 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-3992-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.17 server_addrs= [ [ (*_iter6) = 10.20.0.17:5269, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.17 server_addrs= [ [ (*_iter6) = 10.20.0.17:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.254:8086, ] ] status = Initializing description = Idle : EvIdleHoldTimerExpired -> Connect ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.17, XMPP:dns-server:10.20.0.17, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:11:56:813.759 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Wrong state: Connect for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:11:56:813.786 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Processing scm::EvTcpConnectFail in state Connect 2025-09-01 Mon 02:11:56:813.806 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Connect : EvTcpConnectFail 2025-09-01 Mon 02:11:56:813.903 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: TCP [SYS_ERR]: TcpSessionMessageLog: Session -(-1) > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:11:56:813.983 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Idle 2025-09-01 Mon 02:11:56:814.124 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:56:814.866 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: SANDESH: Send FAILED: 1756692716813574 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3992-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Connect successful_connections = 0 pid = 111888 http_port = 8085 start_time = 1756692145743123 collector_name = collector_ip = 10.0.0.254:8086 collector_list= [ [ (*_iter40) = 10.0.0.254:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ComputeCpuStateTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 9 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 51 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 86 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 132 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterControlStatsTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 19 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 28 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 184 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 47 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 182, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 382, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:11:56:814.926 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:11:56:814.993 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:56:815.046 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: SANDESH: Send FAILED: 1756692716813855 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-3992-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.17 server_addrs= [ [ (*_iter6) = 10.20.0.17:5269, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.17 server_addrs= [ [ (*_iter6) = 10.20.0.17:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.17, XMPP:dns-server:10.20.0.17, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:11:56:815.065 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:11:56:815.087 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Processing scm::EvTcpDeleteSession in state Idle 2025-09-01 Mon 02:11:56:815.144 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:56:815.658 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: SANDESH: Send FAILED: 1756692716814099 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3992-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Idle successful_connections = 0 pid = 111888 http_port = 8085 start_time = 1756692145743123 collector_name = collector_ip = 10.0.0.254:8086 collector_list= [ [ (*_iter40) = 10.0.0.254:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ComputeCpuStateTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 9 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 52 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 86 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 133 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterControlStatsTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 19 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 28 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 184 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 47 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 183, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 383, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:11:56:815.701 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:11:57:143.470 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:57:143.608 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: SANDESH: Send FAILED: 1756692717143411 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.17 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:57:143.663 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:57:143.691 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:57:143.710 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848886536832, Pid 111888]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.17 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:11:57:143.727 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: SANDESH: Send FAILED: 1756692717143635 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.17 state = Connect event = Connect Event local_port = 45253 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:57:143.818 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848866952704, Pid 111888]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:11:57:143.820 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:57:144.059 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:57:144.109 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: SANDESH: Send FAILED: 1756692717143946 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.17 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:57:144.133 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:11:57:144.157 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:11:57:144.186 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: SANDESH: Send FAILED: 1756692717143953 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.17 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:11:57:144.205 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:09:665.454 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848886536832, Pid 111888]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.0.0.254:8085::10.0.0.254:33160 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 255 2025-09-01 Mon 02:12:12:187.630 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:12:187.779 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: SANDESH: Send FAILED: 1756692732187566 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.17 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:12:187.806 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:12:188.041 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848862754304, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:12:188.088 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848886536832, Pid 111888]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.17 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:12:12:188.149 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848862754304, Pid 111888]: SANDESH: Send FAILED: 1756692732187974 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.17 state = Connect event = Connect Event local_port = 33847 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:12:188.281 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848862754304, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:12:188.354 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:12:12:188.580 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848866952704, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:12:188.669 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848866952704, Pid 111888]: SANDESH: Send FAILED: 1756692732188426 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.17 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:12:188.695 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848866952704, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:12:188.720 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848866952704, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:12:188.750 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848866952704, Pid 111888]: SANDESH: Send FAILED: 1756692732188474 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.17 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:12:188.768 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848866952704, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:22:788.773 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848886536832, Pid 111888]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.0.0.254:8085::10.0.0.254:58956 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 255 2025-09-01 Mon 02:12:25:755.786 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:25:756.108 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: SANDESH: Send FAILED: 1756692745755300 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3992-1:Compute:contrail-vrouter-agent:0 sm_queue_count = 0 sm_stats= [ ev_stats= [ [ [ event = scm::EvCollectorUpdate enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvIdleHoldTimerExpired enqueues = 23 dequeues = 23 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvSandeshSend enqueues = 394 dequeues = 394 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvStart enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpConnectFail enqueues = 24 dequeues = 24 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpDeleteSession enqueues = 24 dequeues = 24 enqueue_fails = 0 dequeue_fails = 0 ], [ event = enqueues = 467 dequeues = 467 enqueue_fails = 0 dequeue_fails = 0 ], ] ] state = Idle state_since = 1756692716813980 last_state = Idle last_event = scm::EvSandeshSend last_event_at = 1756692732188716 ] max_sm_queue_count = 6 ] 2025-09-01 Mon 02:12:25:756.186 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:12:25:756.215 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:25:756.632 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: SANDESH: Send FAILED: 1756692745755600 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3992-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Idle successful_connections = 0 pid = 111888 http_port = 8085 start_time = 1756692145743123 collector_name = collector_ip = 10.0.0.254:8086 collector_list= [ [ (*_iter40) = 10.0.0.254:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ComputeCpuStateTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 9 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 53 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 88 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 137 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterControlStatsTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 19 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 28 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 192 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 49 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 189, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 394, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:12:25:756.691 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:12:25:760.283 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:25:760.447 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: SANDESH: Send FAILED: 1756692745760242 [SYS_NOTICE]: VrouterStats: data= [ name = cn-jenkins-deploy-platform-ansible-os-3992-1 cpu_info= [ num_cpu = 8 sys_mem_info= [ total = 32 used = 0 free = 0 buffers = 0 cached = 510398136 node_type = ] meminfo= [ virt = 32561 peakvirt = 0 res = 0 ] cpuload= [ one_min_avg = 0.20125 five_min_avg = 0.20625 fifteen_min_avg = 0.2175 ] cpu_share = 0.3 ] ] 2025-09-01 Mon 02:12:25:760.488 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: VrouterStats 2025-09-01 Mon 02:12:25:760.524 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:25:760.621 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: SANDESH: Send FAILED: 1756692745760271 [SYS_NOTICE]: ComputeCpuStateTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3992-1 cpu_info= [ [ [ mem_virt = 32561 cpu_share = 0.3 used_sys_mem = 3784569160 one_min_cpuload = 0.20125 mem_res = 0 ], ] ] ] 2025-09-01 Mon 02:12:25:760.656 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: ComputeCpuStateTrace 2025-09-01 Mon 02:12:25:760.691 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:25:761.322 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: SANDESH: Send FAILED: 1756692745760330 [SYS_NOTICE]: VrouterStats: data= [ name = cn-jenkins-deploy-platform-ansible-os-3992-1 in_tpkts = 187 in_bytes = 8050 out_tpkts = 207 out_bytes = 9606 raw_vhost_stats= [ name = vhost0 in_pkts = 187 in_bytes = 8050 out_pkts = 207 out_bytes = 9606 speed = -1 duplexity = -1 ] raw_vhost_drop_stats= [ ds_discard = 0 ds_pull = 0 ds_invalid_if = 0 ds_invalid_arp = 0 ds_trap_no_if = 0 ds_nowhere_to_go = 0 ds_flow_queue_limit_exceeded = 0 ds_flow_no_memory = 0 ds_flow_invalid_protocol = 0 ds_flow_nat_no_rflow = 0 ds_flow_action_drop = 0 ds_flow_action_invalid = 0 ds_flow_unusable = 0 ds_flow_table_full = 0 ds_interface_tx_discard = 0 ds_interface_drop = 0 ds_duplicated = 0 ds_push = 0 ds_ttl_exceeded = 0 ds_invalid_nh = 0 ds_invalid_label = 0 ds_invalid_protocol = 0 ds_interface_rx_discard = 0 ds_invalid_mcast_source = 0 ds_head_alloc_fail = 0 ds_pcow_fail = 0 ds_mcast_clone_fail = 0 ds_rewrite_fail = 0 ds_misc = 0 ds_invalid_packet = 0 ds_cksum_err = 0 ds_no_fmd = 0 ds_invalid_vnid = 0 ds_frag_err = 0 ds_invalid_source = 0 ds_mcast_df_bit = 0 ds_l2_no_route = 0 ds_vlan_fwd_tx = 0 ds_vlan_fwd_enq = 0 ds_drop_new_flow = 0 ds_flow_evict = 0 ds_trap_original = 0 ds_no_memory = 0 ds_fragment_queue_fail = 0 ds_no_frag_entry = 0 ds_icmp_error = 0 ds_clone_fail = 0 ds_drop_pkts = 0 ] raw_drop_stats= [ ds_discard = 0 ds_pull = 0 ds_invalid_if = 0 ds_invalid_arp = 0 ds_trap_no_if = 0 ds_nowhere_to_go = 0 ds_flow_queue_limit_exceeded = 0 ds_flow_no_memory = 0 ds_flow_invalid_protocol = 0 ds_flow_nat_no_rflow = 0 ds_flow_action_drop = 0 ds_flow_action_invalid = 0 ds_flow_unusable = 0 ds_flow_table_full = 0 ds_interface_tx_discard = 0 ds_interface_drop = 0 ds_duplicated = 0 ds_push = 0 ds_ttl_exceeded = 0 ds_invalid_nh = 0 ds_invalid_label = 0 ds_invalid_protocol = 0 ds_interface_rx_discard = 0 ds_invalid_mcast_source = 0 ds_head_alloc_fail = 0 ds_pcow_fail = 0 ds_mcast_clone_fail = 0 ds_rewrite_fail = 0 ds_misc = 0 ds_invalid_packet = 0 ds_cksum_err = 0 ds_no_fmd = 0 ds_invalid_vnid = 0 ds_frag_err = 0 ds_invalid_source = 0 ds_mcast_df_bit = 0 ds_l2_no_route = 0 ds_vlan_fwd_tx = 0 ds_vlan_fwd_enq = 0 ds_drop_new_flow = 0 ds_flow_evict = 0 ds_trap_original = 0 ds_no_memory = 0 ds_fragment_queue_fail = 0 ds_no_frag_entry = 0 ds_icmp_error = 0 ds_clone_fail = 0 ds_drop_pkts = 0 ] flow_rate= [ added_flows = 0 max_flow_adds_per_second = 0 min_flow_adds_per_second = 0 deleted_flows = 0 max_flow_deletes_per_second = 0 min_flow_deletes_per_second = 0 active_flows = 0 hold_flows = 0 ] raw_phy_if_stats= [ [ _iter262->first = ens6 [ in_pkts = 27 in_bytes = 2046 out_pkts = 187 out_bytes = 8050 ], ] ] raw_phy_if_drop_stats= [ [ _iter263->first = ens6 [ ds_discard = 0 ds_pull = 0 ds_invalid_if = 0 ds_invalid_arp = 0 ds_trap_no_if = 0 ds_nowhere_to_go = 0 ds_flow_queue_limit_exceeded = 0 ds_flow_no_memory = 0 ds_flow_invalid_protocol = 0 ds_flow_nat_no_rflow = 0 ds_flow_action_drop = 0 ds_flow_action_invalid = 0 ds_flow_unusable = 0 ds_flow_table_full = 0 ds_interface_tx_discard = 0 ds_interface_drop = 0 ds_duplicated = 0 ds_push = 0 ds_ttl_exceeded = 0 ds_invalid_nh = 0 ds_invalid_label = 0 ds_invalid_protocol = 0 ds_interface_rx_discard = 0 ds_invalid_mcast_source = 0 ds_head_alloc_fail = 0 ds_pcow_fail = 0 ds_mcast_clone_fail = 0 ds_rewrite_fail = 0 ds_misc = 0 ds_invalid_packet = 0 ds_cksum_err = 0 ds_no_fmd = 0 ds_invalid_vnid = 0 ds_frag_err = 0 ds_invalid_source = 0 ds_mcast_df_bit = 0 ds_l2_no_route = 0 ds_vlan_fwd_tx = 0 ds_vlan_fwd_enq = 0 ds_drop_new_flow = 0 ds_flow_evict = 0 ds_trap_original = 0 ds_no_memory = 0 ds_fragment_queue_fail = 0 ds_no_frag_entry = 0 ds_icmp_error = 0 ds_clone_fail = 0 ds_drop_pkts = 0 ], ] ] ] 2025-09-01 Mon 02:12:25:761.400 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: VrouterStats 2025-09-01 Mon 02:12:25:761.446 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:25:761.612 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: SANDESH: Send FAILED: 1756692745760406 [SYS_NOTICE]: VrouterControlStatsTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3992-1 raw_xmpp_stats= [ [ _iter306->first = 10.20.0.17 [ reconnects = 0 in_msgs = 0 out_msgs = 0 ], ] ] raw_ifmap_stats= [ [ _iter307->first = link_delete_parse_errors _iter307->second = 0, _iter307->first = link_update_parse_errors _iter307->second = 0, _iter307->first = node_delete_parse_errors _iter307->second = 0, _iter307->first = node_update_parse_errors _iter307->second = 0, ] ] raw_rt_table_size= [ [ _iter308->first = default-domain:default-project:ip-fabric:__default__ [ inet4_unicast = 4 inet4_multicast = 0 evpn = 0 bridge = 2 inet6_unicast = 2 ], _iter308->first = default-domain:default-project:ip-fabric:ip-fabric [ inet4_unicast = 2 inet4_multicast = 0 evpn = 1 bridge = 2 inet6_unicast = 0 ], ] ] ] 2025-09-01 Mon 02:12:25:761.698 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: VrouterControlStatsTrace 2025-09-01 Mon 02:12:26:814.384 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848858555904, Pid 111888]: Processing scm::EvIdleHoldTimerExpired in state Idle 2025-09-01 Mon 02:12:26:814.842 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848858555904, Pid 111888]: Connect : Start Connect timer 10.0.0.254:8086 2025-09-01 Mon 02:12:26:814.916 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848886536832, Pid 111888]: -(-1) OnSessionEvent TCP Connect Failed 2025-09-01 Mon 02:12:26:815.019 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848886536832, Pid 111888]: -(-1):EvTcpConnectFail 2025-09-01 Mon 02:12:26:815.077 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848858555904, Pid 111888]: Processing scm::EvSandeshSend in state Connect 2025-09-01 Mon 02:12:26:815.113 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848858555904, Pid 111888]: SANDESH: Send FAILED: 1756692746814599 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-3992-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.17 server_addrs= [ [ (*_iter6) = 10.20.0.17:5269, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.17 server_addrs= [ [ (*_iter6) = 10.20.0.17:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.254:8086, ] ] status = Initializing description = Idle : EvIdleHoldTimerExpired -> Connect ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.17, XMPP:dns-server:10.20.0.17, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:12:26:815.124 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848858555904, Pid 111888]: Wrong state: Connect for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:12:26:815.137 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848858555904, Pid 111888]: Processing scm::EvTcpConnectFail in state Connect 2025-09-01 Mon 02:12:26:815.145 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848858555904, Pid 111888]: Connect : EvTcpConnectFail 2025-09-01 Mon 02:12:26:815.196 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848858555904, Pid 111888]: TCP [SYS_ERR]: TcpSessionMessageLog: Session -(-1) > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:12:26:815.230 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848858555904, Pid 111888]: Idle 2025-09-01 Mon 02:12:26:815.289 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848858555904, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:26:815.585 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848858555904, Pid 111888]: SANDESH: Send FAILED: 1756692746815048 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3992-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Connect successful_connections = 0 pid = 111888 http_port = 8085 start_time = 1756692145743123 collector_name = collector_ip = 10.0.0.254:8086 collector_list= [ [ (*_iter40) = 10.0.0.254:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ComputeCpuStateTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 10 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 53 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 90 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 137 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterControlStatsTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 20 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 30 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 192 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 49 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 189, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 400, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:12:26:815.629 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848858555904, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:12:26:815.645 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848858555904, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:26:815.667 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848858555904, Pid 111888]: SANDESH: Send FAILED: 1756692746815170 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-3992-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.17 server_addrs= [ [ (*_iter6) = 10.20.0.17:5269, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.17 server_addrs= [ [ (*_iter6) = 10.20.0.17:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.17, XMPP:dns-server:10.20.0.17, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:12:26:815.675 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848858555904, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:12:26:815.726 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848858555904, Pid 111888]: Processing scm::EvTcpDeleteSession in state Idle 2025-09-01 Mon 02:12:26:815.807 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848858555904, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:26:815.991 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848858555904, Pid 111888]: SANDESH: Send FAILED: 1756692746815277 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3992-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Idle successful_connections = 0 pid = 111888 http_port = 8085 start_time = 1756692145743123 collector_name = collector_ip = 10.0.0.254:8086 collector_list= [ [ (*_iter40) = 10.0.0.254:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ComputeCpuStateTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 10 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 54 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 90 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 138 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterControlStatsTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 20 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 30 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 192 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 49 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 190, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 401, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:12:26:816.312 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848858555904, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:12:29:274.404 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848862754304, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:29:274.502 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848886536832, Pid 111888]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.17 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:12:29:274.611 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848862754304, Pid 111888]: SANDESH: Send FAILED: 1756692749274193 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.17 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:29:274.718 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848862754304, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:29:274.758 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848862754304, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:29:274.842 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848862754304, Pid 111888]: SANDESH: Send FAILED: 1756692749274382 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.17 state = Connect event = Connect Event local_port = 36061 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:29:274.884 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848862754304, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:29:274.846 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848858555904, Pid 111888]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:12:29:275.047 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848854357504, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:29:275.085 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848854357504, Pid 111888]: SANDESH: Send FAILED: 1756692749274994 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.17 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:29:275.096 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848854357504, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:29:275.106 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848854357504, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:29:275.116 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848854357504, Pid 111888]: SANDESH: Send FAILED: 1756692749275043 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.17 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:29:275.123 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848854357504, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:33:595.892 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848886536832, Pid 111888]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.20.0.17:8085::10.20.0.17:33082 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 255 2025-09-01 Mon 02:12:33:614.972 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848886536832, Pid 111888]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.20.0.17:8085::10.20.0.17:33088 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 255 2025-09-01 Mon 02:12:35:545.537 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848886536832, Pid 111888]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.0.0.254:8085::10.0.0.254:33786 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 255 2025-09-01 Mon 02:12:42:428.822 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848879547904, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:42:428.889 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848879547904, Pid 111888]: SANDESH: Send FAILED: 1756692762428786 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.17 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:42:428.902 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848879547904, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:42:428.946 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848879547904, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:42:428.963 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848879547904, Pid 111888]: SANDESH: Send FAILED: 1756692762428938 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.17 state = Connect event = Connect Event local_port = 43705 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:42:429.000 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848879547904, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:42:428.999 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848886536832, Pid 111888]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.17 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:12:42:429.195 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848866952704, Pid 111888]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:12:42:429.267 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848854357504, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:42:429.356 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848854357504, Pid 111888]: SANDESH: Send FAILED: 1756692762429232 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.17 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:42:429.383 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848854357504, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:42:429.410 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848854357504, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:42:429.440 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848854357504, Pid 111888]: SANDESH: Send FAILED: 1756692762429237 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.17 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:42:429.459 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848854357504, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:48:570.900 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848886536832, Pid 111888]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.0.0.254:8085::10.0.0.254:43288 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 255 2025-09-01 Mon 02:12:55:755.997 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848862754304, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:55:756.076 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848862754304, Pid 111888]: SANDESH: Send FAILED: 1756692775755870 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3992-1:Compute:contrail-vrouter-agent:0 sm_queue_count = 0 sm_stats= [ ev_stats= [ [ [ event = scm::EvCollectorUpdate enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvIdleHoldTimerExpired enqueues = 24 dequeues = 24 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvSandeshSend enqueues = 412 dequeues = 412 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvStart enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpConnectFail enqueues = 25 dequeues = 25 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpDeleteSession enqueues = 25 dequeues = 25 enqueue_fails = 0 dequeue_fails = 0 ], [ event = enqueues = 488 dequeues = 488 enqueue_fails = 0 dequeue_fails = 0 ], ] ] state = Idle state_since = 1756692746815229 last_state = Idle last_event = scm::EvSandeshSend last_event_at = 1756692762429406 ] max_sm_queue_count = 6 ] 2025-09-01 Mon 02:12:55:756.088 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848862754304, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:12:55:756.108 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848862754304, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:55:756.220 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848862754304, Pid 111888]: SANDESH: Send FAILED: 1756692775755970 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3992-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Idle successful_connections = 0 pid = 111888 http_port = 8085 start_time = 1756692145743123 collector_name = collector_ip = 10.0.0.254:8086 collector_list= [ [ (*_iter40) = 10.0.0.254:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ComputeCpuStateTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 10 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 55 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 92 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 144 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterControlStatsTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 20 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 30 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 200 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 51 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 198, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 412, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:12:55:756.234 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848862754304, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:12:55:760.592 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848879547904, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:55:760.911 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848879547904, Pid 111888]: SANDESH: Send FAILED: 1756692775760538 [SYS_NOTICE]: VrouterStats: data= [ name = cn-jenkins-deploy-platform-ansible-os-3992-1 in_tpkts = 190 in_bytes = 8176 out_tpkts = 210 out_bytes = 9732 raw_vhost_stats= [ name = vhost0 in_pkts = 190 in_bytes = 8176 out_pkts = 210 out_bytes = 9732 speed = -1 duplexity = -1 ] raw_vhost_drop_stats= [ ds_discard = 0 ds_pull = 0 ds_invalid_if = 0 ds_invalid_arp = 0 ds_trap_no_if = 0 ds_nowhere_to_go = 0 ds_flow_queue_limit_exceeded = 0 ds_flow_no_memory = 0 ds_flow_invalid_protocol = 0 ds_flow_nat_no_rflow = 0 ds_flow_action_drop = 0 ds_flow_action_invalid = 0 ds_flow_unusable = 0 ds_flow_table_full = 0 ds_interface_tx_discard = 0 ds_interface_drop = 0 ds_duplicated = 0 ds_push = 0 ds_ttl_exceeded = 0 ds_invalid_nh = 0 ds_invalid_label = 0 ds_invalid_protocol = 0 ds_interface_rx_discard = 0 ds_invalid_mcast_source = 0 ds_head_alloc_fail = 0 ds_pcow_fail = 0 ds_mcast_clone_fail = 0 ds_rewrite_fail = 0 ds_misc = 0 ds_invalid_packet = 0 ds_cksum_err = 0 ds_no_fmd = 0 ds_invalid_vnid = 0 ds_frag_err = 0 ds_invalid_source = 0 ds_mcast_df_bit = 0 ds_l2_no_route = 0 ds_vlan_fwd_tx = 0 ds_vlan_fwd_enq = 0 ds_drop_new_flow = 0 ds_flow_evict = 0 ds_trap_original = 0 ds_no_memory = 0 ds_fragment_queue_fail = 0 ds_no_frag_entry = 0 ds_icmp_error = 0 ds_clone_fail = 0 ds_drop_pkts = 0 ] phy_if_5min_usage= [ [ [ name = ens6 in_bandwidth_usage = 7 out_bandwidth_usage = 35 ], ] ] raw_drop_stats= [ ds_discard = 0 ds_pull = 0 ds_invalid_if = 0 ds_invalid_arp = 0 ds_trap_no_if = 0 ds_nowhere_to_go = 0 ds_flow_queue_limit_exceeded = 0 ds_flow_no_memory = 0 ds_flow_invalid_protocol = 0 ds_flow_nat_no_rflow = 0 ds_flow_action_drop = 0 ds_flow_action_invalid = 0 ds_flow_unusable = 0 ds_flow_table_full = 0 ds_interface_tx_discard = 0 ds_interface_drop = 0 ds_duplicated = 0 ds_push = 0 ds_ttl_exceeded = 0 ds_invalid_nh = 0 ds_invalid_label = 0 ds_invalid_protocol = 0 ds_interface_rx_discard = 0 ds_invalid_mcast_source = 0 ds_head_alloc_fail = 0 ds_pcow_fail = 0 ds_mcast_clone_fail = 0 ds_rewrite_fail = 0 ds_misc = 0 ds_invalid_packet = 0 ds_cksum_err = 0 ds_no_fmd = 0 ds_invalid_vnid = 0 ds_frag_err = 0 ds_invalid_source = 0 ds_mcast_df_bit = 0 ds_l2_no_route = 0 ds_vlan_fwd_tx = 0 ds_vlan_fwd_enq = 0 ds_drop_new_flow = 0 ds_flow_evict = 0 ds_trap_original = 0 ds_no_memory = 0 ds_fragment_queue_fail = 0 ds_no_frag_entry = 0 ds_icmp_error = 0 ds_clone_fail = 0 ds_drop_pkts = 0 ] total_in_bandwidth_utilization = 4.87891e-17 flow_rate= [ added_flows = 0 max_flow_adds_per_second = 0 min_flow_adds_per_second = 0 deleted_flows = 0 max_flow_deletes_per_second = 0 min_flow_deletes_per_second = 0 active_flows = 0 hold_flows = 0 ] phy_band_in_bps= [ [ _iter260->first = ens6 _iter260->second = 9, ] ] phy_band_out_bps= [ [ _iter261->first = ens6 _iter261->second = 33, ] ] raw_phy_if_stats= [ [ _iter262->first = ens6 [ in_pkts = 27 in_bytes = 2046 out_pkts = 190 out_bytes = 8176 ], ] ] raw_phy_if_drop_stats= [ [ _iter263->first = ens6 [ ds_discard = 0 ds_pull = 0 ds_invalid_if = 0 ds_invalid_arp = 0 ds_trap_no_if = 0 ds_nowhere_to_go = 0 ds_flow_queue_limit_exceeded = 0 ds_flow_no_memory = 0 ds_flow_invalid_protocol = 0 ds_flow_nat_no_rflow = 0 ds_flow_action_drop = 0 ds_flow_action_invalid = 0 ds_flow_unusable = 0 ds_flow_table_full = 0 ds_interface_tx_discard = 0 ds_interface_drop = 0 ds_duplicated = 0 ds_push = 0 ds_ttl_exceeded = 0 ds_invalid_nh = 0 ds_invalid_label = 0 ds_invalid_protocol = 0 ds_interface_rx_discard = 0 ds_invalid_mcast_source = 0 ds_head_alloc_fail = 0 ds_pcow_fail = 0 ds_mcast_clone_fail = 0 ds_rewrite_fail = 0 ds_misc = 0 ds_invalid_packet = 0 ds_cksum_err = 0 ds_no_fmd = 0 ds_invalid_vnid = 0 ds_frag_err = 0 ds_invalid_source = 0 ds_mcast_df_bit = 0 ds_l2_no_route = 0 ds_vlan_fwd_tx = 0 ds_vlan_fwd_enq = 0 ds_drop_new_flow = 0 ds_flow_evict = 0 ds_trap_original = 0 ds_no_memory = 0 ds_fragment_queue_fail = 0 ds_no_frag_entry = 0 ds_icmp_error = 0 ds_clone_fail = 0 ds_drop_pkts = 0 ], ] ] ] 2025-09-01 Mon 02:12:55:760.971 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848879547904, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: VrouterStats 2025-09-01 Mon 02:12:55:760.994 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848879547904, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:55:761.058 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848879547904, Pid 111888]: SANDESH: Send FAILED: 1756692775760581 [SYS_NOTICE]: VrouterControlStatsTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3992-1 raw_xmpp_stats= [ [ _iter306->first = 10.20.0.17 [ reconnects = 0 in_msgs = 0 out_msgs = 0 ], ] ] raw_ifmap_stats= [ [ _iter307->first = link_delete_parse_errors _iter307->second = 0, _iter307->first = link_update_parse_errors _iter307->second = 0, _iter307->first = node_delete_parse_errors _iter307->second = 0, _iter307->first = node_update_parse_errors _iter307->second = 0, ] ] raw_rt_table_size= [ [ _iter308->first = default-domain:default-project:ip-fabric:__default__ [ inet4_unicast = 4 inet4_multicast = 0 evpn = 0 bridge = 2 inet6_unicast = 2 ], _iter308->first = default-domain:default-project:ip-fabric:ip-fabric [ inet4_unicast = 2 inet4_multicast = 0 evpn = 1 bridge = 2 inet6_unicast = 0 ], ] ] ] 2025-09-01 Mon 02:12:55:761.072 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848879547904, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: VrouterControlStatsTrace 2025-09-01 Mon 02:12:56:815.517 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: Processing scm::EvIdleHoldTimerExpired in state Idle 2025-09-01 Mon 02:12:56:816.096 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: Connect : Start Connect timer 10.0.0.254:8086 2025-09-01 Mon 02:12:56:816.175 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848886536832, Pid 111888]: -(-1) OnSessionEvent TCP Connect Failed 2025-09-01 Mon 02:12:56:816.258 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848886536832, Pid 111888]: -(-1):EvTcpConnectFail 2025-09-01 Mon 02:12:56:816.359 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: Processing scm::EvSandeshSend in state Connect 2025-09-01 Mon 02:12:56:816.431 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: SANDESH: Send FAILED: 1756692776815719 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-3992-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.17 server_addrs= [ [ (*_iter6) = 10.20.0.17:5269, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.17 server_addrs= [ [ (*_iter6) = 10.20.0.17:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.254:8086, ] ] status = Initializing description = Idle : EvIdleHoldTimerExpired -> Connect ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.17, XMPP:dns-server:10.20.0.17, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:12:56:816.478 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: Wrong state: Connect for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:12:56:816.505 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: Processing scm::EvTcpConnectFail in state Connect 2025-09-01 Mon 02:12:56:816.526 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: Connect : EvTcpConnectFail 2025-09-01 Mon 02:12:56:816.673 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: TCP [SYS_ERR]: TcpSessionMessageLog: Session -(-1) > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:12:56:816.744 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: Idle 2025-09-01 Mon 02:12:56:816.937 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:56:817.340 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: SANDESH: Send FAILED: 1756692776816305 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3992-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Connect successful_connections = 0 pid = 111888 http_port = 8085 start_time = 1756692145743123 collector_name = collector_ip = 10.0.0.254:8086 collector_list= [ [ (*_iter40) = 10.0.0.254:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ComputeCpuStateTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 10 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 55 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 94 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 144 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterControlStatsTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 21 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 31 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 200 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 51 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 198, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 416, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:12:56:817.397 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:12:56:817.427 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:56:817.477 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: SANDESH: Send FAILED: 1756692776816619 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-3992-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.17 server_addrs= [ [ (*_iter6) = 10.20.0.17:5269, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.17 server_addrs= [ [ (*_iter6) = 10.20.0.17:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.17, XMPP:dns-server:10.20.0.17, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:12:56:817.496 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:12:56:817.519 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: Processing scm::EvTcpDeleteSession in state Idle 2025-09-01 Mon 02:12:56:817.615 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:56:818.085 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: SANDESH: Send FAILED: 1756692776816856 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3992-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Idle successful_connections = 0 pid = 111888 http_port = 8085 start_time = 1756692145743123 collector_name = collector_ip = 10.0.0.254:8086 collector_list= [ [ (*_iter40) = 10.0.0.254:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ComputeCpuStateTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 10 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 56 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 94 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 145 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterControlStatsTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 21 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 31 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 200 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 51 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 199, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 417, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:12:56:818.138 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:12:58:165.427 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:58:165.492 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: SANDESH: Send FAILED: 1756692778165375 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.17 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:58:165.503 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:58:165.892 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848886536832, Pid 111888]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.17 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:12:58:165.902 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848879547904, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:58:166.064 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848862754304, Pid 111888]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:12:58:166.221 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848879547904, Pid 111888]: SANDESH: Send FAILED: 1756692778165816 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.17 state = Connect event = Connect Event local_port = 42335 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:58:166.375 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848879547904, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:58:166.417 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848879547904, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:58:166.469 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848879547904, Pid 111888]: SANDESH: Send FAILED: 1756692778166359 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.17 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:58:166.500 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848879547904, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:12:58:166.533 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848879547904, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:12:58:166.656 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848879547904, Pid 111888]: SANDESH: Send FAILED: 1756692778166380 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.17 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:12:58:166.691 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848879547904, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:01:531.157 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848886536832, Pid 111888]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.0.0.254:8085::10.0.0.254:53556 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 255 2025-09-01 Mon 02:13:13:209.765 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848862754304, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:13:209.880 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848862754304, Pid 111888]: SANDESH: Send FAILED: 1756692793209605 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.17 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:13:209.908 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848862754304, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:13:209.931 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848886536832, Pid 111888]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.17 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:13:13:209.938 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848862754304, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:13:210.050 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848862754304, Pid 111888]: SANDESH: Send FAILED: 1756692793209864 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.17 state = Connect event = Connect Event local_port = 37107 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:13:210.058 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848858555904, Pid 111888]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:13:13:210.084 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848862754304, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:13:210.157 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848862754304, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:13:210.185 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848862754304, Pid 111888]: SANDESH: Send FAILED: 1756692793210137 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.17 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:13:210.203 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848862754304, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:13:210.224 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848862754304, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:13:210.250 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848862754304, Pid 111888]: SANDESH: Send FAILED: 1756692793210145 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.17 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:13:210.289 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848862754304, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:14:383.805 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848886536832, Pid 111888]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.0.0.254:8085::10.0.0.254:32826 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 255 2025-09-01 Mon 02:13:25:756.347 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:25:756.425 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: SANDESH: Send FAILED: 1756692805756198 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3992-1:Compute:contrail-vrouter-agent:0 sm_queue_count = 0 sm_stats= [ ev_stats= [ [ [ event = scm::EvCollectorUpdate enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvIdleHoldTimerExpired enqueues = 25 dequeues = 25 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvSandeshSend enqueues = 428 dequeues = 428 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvStart enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpConnectFail enqueues = 26 dequeues = 26 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpDeleteSession enqueues = 26 dequeues = 26 enqueue_fails = 0 dequeue_fails = 0 ], [ event = enqueues = 507 dequeues = 507 enqueue_fails = 0 dequeue_fails = 0 ], ] ] state = Idle state_since = 1756692776816742 last_state = Idle last_event = scm::EvSandeshSend last_event_at = 1756692793210220 ] max_sm_queue_count = 6 ] 2025-09-01 Mon 02:13:25:756.439 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:13:25:756.448 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:25:756.596 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: SANDESH: Send FAILED: 1756692805756300 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3992-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Idle successful_connections = 0 pid = 111888 http_port = 8085 start_time = 1756692145743123 collector_name = collector_ip = 10.0.0.254:8086 collector_list= [ [ (*_iter40) = 10.0.0.254:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ComputeCpuStateTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 10 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 57 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 96 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 149 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterControlStatsTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 21 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 31 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 208 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 53 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 205, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 428, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:13:25:756.621 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:13:25:760.872 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848879547904, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:25:761.022 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848879547904, Pid 111888]: SANDESH: Send FAILED: 1756692805760818 [SYS_NOTICE]: VrouterStats: data= [ name = cn-jenkins-deploy-platform-ansible-os-3992-1 cpu_info= [ num_cpu = 8 sys_mem_info= [ total = 32 used = 0 free = 0 buffers = 0 cached = 510398136 node_type = ] meminfo= [ virt = 32561 peakvirt = 0 res = 0 ] cpuload= [ one_min_avg = 0.16625 five_min_avg = 0.1975 fifteen_min_avg = 0.21375 ] cpu_share = 0.285417 ] ] 2025-09-01 Mon 02:13:25:761.051 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848879547904, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: VrouterStats 2025-09-01 Mon 02:13:25:761.079 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848879547904, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:25:761.148 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848879547904, Pid 111888]: SANDESH: Send FAILED: 1756692805760836 [SYS_NOTICE]: ComputeCpuStateTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3992-1 cpu_info= [ [ [ mem_virt = 32561 cpu_share = 0.285417 used_sys_mem = 3784569160 one_min_cpuload = 0.16625 mem_res = 0 ], ] ] ] 2025-09-01 Mon 02:13:25:761.170 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848879547904, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: ComputeCpuStateTrace 2025-09-01 Mon 02:13:25:761.203 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848879547904, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:25:761.624 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848879547904, Pid 111888]: SANDESH: Send FAILED: 1756692805760867 [SYS_NOTICE]: VrouterStats: data= [ name = cn-jenkins-deploy-platform-ansible-os-3992-1 in_tpkts = 193 in_bytes = 8302 out_tpkts = 213 out_bytes = 9858 raw_vhost_stats= [ name = vhost0 in_pkts = 193 in_bytes = 8302 out_pkts = 213 out_bytes = 9858 speed = -1 duplexity = -1 ] raw_vhost_drop_stats= [ ds_discard = 0 ds_pull = 0 ds_invalid_if = 0 ds_invalid_arp = 0 ds_trap_no_if = 0 ds_nowhere_to_go = 0 ds_flow_queue_limit_exceeded = 0 ds_flow_no_memory = 0 ds_flow_invalid_protocol = 0 ds_flow_nat_no_rflow = 0 ds_flow_action_drop = 0 ds_flow_action_invalid = 0 ds_flow_unusable = 0 ds_flow_table_full = 0 ds_interface_tx_discard = 0 ds_interface_drop = 0 ds_duplicated = 0 ds_push = 0 ds_ttl_exceeded = 0 ds_invalid_nh = 0 ds_invalid_label = 0 ds_invalid_protocol = 0 ds_interface_rx_discard = 0 ds_invalid_mcast_source = 0 ds_head_alloc_fail = 0 ds_pcow_fail = 0 ds_mcast_clone_fail = 0 ds_rewrite_fail = 0 ds_misc = 0 ds_invalid_packet = 0 ds_cksum_err = 0 ds_no_fmd = 0 ds_invalid_vnid = 0 ds_frag_err = 0 ds_invalid_source = 0 ds_mcast_df_bit = 0 ds_l2_no_route = 0 ds_vlan_fwd_tx = 0 ds_vlan_fwd_enq = 0 ds_drop_new_flow = 0 ds_flow_evict = 0 ds_trap_original = 0 ds_no_memory = 0 ds_fragment_queue_fail = 0 ds_no_frag_entry = 0 ds_icmp_error = 0 ds_clone_fail = 0 ds_drop_pkts = 0 ] raw_drop_stats= [ ds_discard = 0 ds_pull = 0 ds_invalid_if = 0 ds_invalid_arp = 0 ds_trap_no_if = 0 ds_nowhere_to_go = 0 ds_flow_queue_limit_exceeded = 0 ds_flow_no_memory = 0 ds_flow_invalid_protocol = 0 ds_flow_nat_no_rflow = 0 ds_flow_action_drop = 0 ds_flow_action_invalid = 0 ds_flow_unusable = 0 ds_flow_table_full = 0 ds_interface_tx_discard = 0 ds_interface_drop = 0 ds_duplicated = 0 ds_push = 0 ds_ttl_exceeded = 0 ds_invalid_nh = 0 ds_invalid_label = 0 ds_invalid_protocol = 0 ds_interface_rx_discard = 0 ds_invalid_mcast_source = 0 ds_head_alloc_fail = 0 ds_pcow_fail = 0 ds_mcast_clone_fail = 0 ds_rewrite_fail = 0 ds_misc = 0 ds_invalid_packet = 0 ds_cksum_err = 0 ds_no_fmd = 0 ds_invalid_vnid = 0 ds_frag_err = 0 ds_invalid_source = 0 ds_mcast_df_bit = 0 ds_l2_no_route = 0 ds_vlan_fwd_tx = 0 ds_vlan_fwd_enq = 0 ds_drop_new_flow = 0 ds_flow_evict = 0 ds_trap_original = 0 ds_no_memory = 0 ds_fragment_queue_fail = 0 ds_no_frag_entry = 0 ds_icmp_error = 0 ds_clone_fail = 0 ds_drop_pkts = 0 ] flow_rate= [ added_flows = 0 max_flow_adds_per_second = 0 min_flow_adds_per_second = 0 deleted_flows = 0 max_flow_deletes_per_second = 0 min_flow_deletes_per_second = 0 active_flows = 0 hold_flows = 0 ] raw_phy_if_stats= [ [ _iter262->first = ens6 [ in_pkts = 27 in_bytes = 2046 out_pkts = 193 out_bytes = 8302 ], ] ] raw_phy_if_drop_stats= [ [ _iter263->first = ens6 [ ds_discard = 0 ds_pull = 0 ds_invalid_if = 0 ds_invalid_arp = 0 ds_trap_no_if = 0 ds_nowhere_to_go = 0 ds_flow_queue_limit_exceeded = 0 ds_flow_no_memory = 0 ds_flow_invalid_protocol = 0 ds_flow_nat_no_rflow = 0 ds_flow_action_drop = 0 ds_flow_action_invalid = 0 ds_flow_unusable = 0 ds_flow_table_full = 0 ds_interface_tx_discard = 0 ds_interface_drop = 0 ds_duplicated = 0 ds_push = 0 ds_ttl_exceeded = 0 ds_invalid_nh = 0 ds_invalid_label = 0 ds_invalid_protocol = 0 ds_interface_rx_discard = 0 ds_invalid_mcast_source = 0 ds_head_alloc_fail = 0 ds_pcow_fail = 0 ds_mcast_clone_fail = 0 ds_rewrite_fail = 0 ds_misc = 0 ds_invalid_packet = 0 ds_cksum_err = 0 ds_no_fmd = 0 ds_invalid_vnid = 0 ds_frag_err = 0 ds_invalid_source = 0 ds_mcast_df_bit = 0 ds_l2_no_route = 0 ds_vlan_fwd_tx = 0 ds_vlan_fwd_enq = 0 ds_drop_new_flow = 0 ds_flow_evict = 0 ds_trap_original = 0 ds_no_memory = 0 ds_fragment_queue_fail = 0 ds_no_frag_entry = 0 ds_icmp_error = 0 ds_clone_fail = 0 ds_drop_pkts = 0 ], ] ] ] 2025-09-01 Mon 02:13:25:761.712 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848879547904, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: VrouterStats 2025-09-01 Mon 02:13:25:761.739 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848879547904, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:25:761.811 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848879547904, Pid 111888]: SANDESH: Send FAILED: 1756692805760906 [SYS_NOTICE]: VrouterControlStatsTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3992-1 raw_xmpp_stats= [ [ _iter306->first = 10.20.0.17 [ reconnects = 0 in_msgs = 0 out_msgs = 0 ], ] ] raw_ifmap_stats= [ [ _iter307->first = link_delete_parse_errors _iter307->second = 0, _iter307->first = link_update_parse_errors _iter307->second = 0, _iter307->first = node_delete_parse_errors _iter307->second = 0, _iter307->first = node_update_parse_errors _iter307->second = 0, ] ] raw_rt_table_size= [ [ _iter308->first = default-domain:default-project:ip-fabric:__default__ [ inet4_unicast = 4 inet4_multicast = 0 evpn = 0 bridge = 2 inet6_unicast = 2 ], _iter308->first = default-domain:default-project:ip-fabric:ip-fabric [ inet4_unicast = 2 inet4_multicast = 0 evpn = 1 bridge = 2 inet6_unicast = 0 ], ] ] ] 2025-09-01 Mon 02:13:25:761.832 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848879547904, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: VrouterControlStatsTrace 2025-09-01 Mon 02:13:26:816.879 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: Processing scm::EvIdleHoldTimerExpired in state Idle 2025-09-01 Mon 02:13:26:817.160 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: Connect : Start Connect timer 10.0.0.254:8086 2025-09-01 Mon 02:13:26:817.192 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848886536832, Pid 111888]: -(-1) OnSessionEvent TCP Connect Failed 2025-09-01 Mon 02:13:26:817.240 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848886536832, Pid 111888]: -(-1):EvTcpConnectFail 2025-09-01 Mon 02:13:26:817.299 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: Processing scm::EvSandeshSend in state Connect 2025-09-01 Mon 02:13:26:817.329 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: SANDESH: Send FAILED: 1756692806816961 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-3992-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.17 server_addrs= [ [ (*_iter6) = 10.20.0.17:5269, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.17 server_addrs= [ [ (*_iter6) = 10.20.0.17:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.254:8086, ] ] status = Initializing description = Idle : EvIdleHoldTimerExpired -> Connect ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.17, XMPP:dns-server:10.20.0.17, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:13:26:817.338 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: Wrong state: Connect for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:13:26:817.347 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: Processing scm::EvTcpConnectFail in state Connect 2025-09-01 Mon 02:13:26:817.362 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: Connect : EvTcpConnectFail 2025-09-01 Mon 02:13:26:817.404 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: TCP [SYS_ERR]: TcpSessionMessageLog: Session -(-1) > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:13:26:817.434 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: Idle 2025-09-01 Mon 02:13:26:817.499 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:26:817.698 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: SANDESH: Send FAILED: 1756692806817266 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3992-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Connect successful_connections = 0 pid = 111888 http_port = 8085 start_time = 1756692145743123 collector_name = collector_ip = 10.0.0.254:8086 collector_list= [ [ (*_iter40) = 10.0.0.254:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ComputeCpuStateTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 11 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 57 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 98 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 149 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterControlStatsTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 22 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 33 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 208 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 53 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 205, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 434, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:13:26:817.721 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:13:26:817.731 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:26:817.748 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: SANDESH: Send FAILED: 1756692806817384 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-3992-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.17 server_addrs= [ [ (*_iter6) = 10.20.0.17:5269, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.17 server_addrs= [ [ (*_iter6) = 10.20.0.17:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.17, XMPP:dns-server:10.20.0.17, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:13:26:817.754 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:13:26:817.762 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: Processing scm::EvTcpDeleteSession in state Idle 2025-09-01 Mon 02:13:26:817.787 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:26:817.910 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: SANDESH: Send FAILED: 1756692806817474 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3992-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Idle successful_connections = 0 pid = 111888 http_port = 8085 start_time = 1756692145743123 collector_name = collector_ip = 10.0.0.254:8086 collector_list= [ [ (*_iter40) = 10.0.0.254:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ComputeCpuStateTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 11 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 58 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 98 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 150 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterControlStatsTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 22 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 33 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 208 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 53 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 206, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 435, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:13:26:817.928 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:13:27:326.721 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:27:326.784 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: SANDESH: Send FAILED: 1756692807326690 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.17 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:27:326.794 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:27:326.916 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848858555904, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:27:326.963 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848858555904, Pid 111888]: SANDESH: Send FAILED: 1756692807326881 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.17 state = Connect event = Connect Event local_port = 42419 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:27:326.980 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848858555904, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:27:326.964 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848886536832, Pid 111888]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.17 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:13:27:327.080 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848858555904, Pid 111888]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:13:27:327.213 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:27:327.298 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: SANDESH: Send FAILED: 1756692807327108 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.17 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:27:327.334 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:27:327.371 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:27:327.415 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: SANDESH: Send FAILED: 1756692807327143 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.17 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:27:327.436 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:27:735.327 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848886536832, Pid 111888]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.0.0.254:8085::10.0.0.254:59394 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 255 2025-09-01 Mon 02:13:34:593.860 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848886536832, Pid 111888]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.20.0.17:8085::10.20.0.17:49958 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 255 2025-09-01 Mon 02:13:34:613.022 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848886536832, Pid 111888]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.20.0.17:8085::10.20.0.17:49974 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 255 2025-09-01 Mon 02:13:40:605.614 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848886536832, Pid 111888]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.0.0.254:8085::10.0.0.254:43714 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 255 2025-09-01 Mon 02:13:42:370.461 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848866952704, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:42:370.606 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848866952704, Pid 111888]: SANDESH: Send FAILED: 1756692822370410 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.17 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:42:370.637 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848866952704, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:42:370.789 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:42:370.916 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: SANDESH: Send FAILED: 1756692822370720 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.17 state = Connect event = Connect Event local_port = 45249 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:42:370.941 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:42:371.248 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848886536832, Pid 111888]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.17 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:13:42:371.450 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:13:42:371.588 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848858555904, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:42:371.679 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848858555904, Pid 111888]: SANDESH: Send FAILED: 1756692822371508 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.17 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:42:371.711 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848858555904, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:42:371.737 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848858555904, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:42:371.766 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848858555904, Pid 111888]: SANDESH: Send FAILED: 1756692822371520 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.17 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:42:371.784 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848858555904, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:53:399.032 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848886536832, Pid 111888]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.0.0.254:8085::10.0.0.254:40270 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 255 2025-09-01 Mon 02:13:55:137.489 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:55:137.599 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: SANDESH: Send FAILED: 1756692835137441 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.17 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:55:137.620 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:55:137.690 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848862754304, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:55:137.755 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848862754304, Pid 111888]: SANDESH: Send FAILED: 1756692835137637 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.17 state = Connect event = Connect Event local_port = 34185 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:55:137.769 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848862754304, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:55:137.800 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848886536832, Pid 111888]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.17 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:13:55:138.025 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848862754304, Pid 111888]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:13:55:138.100 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:55:138.188 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: SANDESH: Send FAILED: 1756692835138063 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.17 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:55:138.227 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:55:138.254 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:55:138.304 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: SANDESH: Send FAILED: 1756692835138070 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.17 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:13:55:138.332 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:13:55:756.836 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848858555904, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:55:756.950 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848858555904, Pid 111888]: SANDESH: Send FAILED: 1756692835756654 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3992-1:Compute:contrail-vrouter-agent:0 sm_queue_count = 0 sm_stats= [ ev_stats= [ [ [ event = scm::EvCollectorUpdate enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvIdleHoldTimerExpired enqueues = 26 dequeues = 26 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvSandeshSend enqueues = 450 dequeues = 450 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvStart enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpConnectFail enqueues = 27 dequeues = 27 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpDeleteSession enqueues = 27 dequeues = 27 enqueue_fails = 0 dequeue_fails = 0 ], [ event = enqueues = 532 dequeues = 532 enqueue_fails = 0 dequeue_fails = 0 ], ] ] state = Idle state_since = 1756692806817433 last_state = Idle last_event = scm::EvSandeshSend last_event_at = 1756692835138250 ] max_sm_queue_count = 6 ] 2025-09-01 Mon 02:13:55:756.976 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848858555904, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:13:55:756.985 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848858555904, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:55:757.270 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848858555904, Pid 111888]: SANDESH: Send FAILED: 1756692835756786 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3992-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Idle successful_connections = 0 pid = 111888 http_port = 8085 start_time = 1756692145743123 collector_name = collector_ip = 10.0.0.254:8086 collector_list= [ [ (*_iter40) = 10.0.0.254:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ComputeCpuStateTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 11 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 59 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 100 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 158 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterControlStatsTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 22 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 33 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 220 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 56 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 217, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 450, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:13:55:757.304 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848858555904, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:13:55:761.327 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848862754304, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:55:761.847 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848862754304, Pid 111888]: SANDESH: Send FAILED: 1756692835761203 [SYS_NOTICE]: VrouterStats: data= [ name = cn-jenkins-deploy-platform-ansible-os-3992-1 in_tpkts = 196 in_bytes = 8428 out_tpkts = 216 out_bytes = 9984 raw_vhost_stats= [ name = vhost0 in_pkts = 196 in_bytes = 8428 out_pkts = 216 out_bytes = 9984 speed = -1 duplexity = -1 ] raw_vhost_drop_stats= [ ds_discard = 0 ds_pull = 0 ds_invalid_if = 0 ds_invalid_arp = 0 ds_trap_no_if = 0 ds_nowhere_to_go = 0 ds_flow_queue_limit_exceeded = 0 ds_flow_no_memory = 0 ds_flow_invalid_protocol = 0 ds_flow_nat_no_rflow = 0 ds_flow_action_drop = 0 ds_flow_action_invalid = 0 ds_flow_unusable = 0 ds_flow_table_full = 0 ds_interface_tx_discard = 0 ds_interface_drop = 0 ds_duplicated = 0 ds_push = 0 ds_ttl_exceeded = 0 ds_invalid_nh = 0 ds_invalid_label = 0 ds_invalid_protocol = 0 ds_interface_rx_discard = 0 ds_invalid_mcast_source = 0 ds_head_alloc_fail = 0 ds_pcow_fail = 0 ds_mcast_clone_fail = 0 ds_rewrite_fail = 0 ds_misc = 0 ds_invalid_packet = 0 ds_cksum_err = 0 ds_no_fmd = 0 ds_invalid_vnid = 0 ds_frag_err = 0 ds_invalid_source = 0 ds_mcast_df_bit = 0 ds_l2_no_route = 0 ds_vlan_fwd_tx = 0 ds_vlan_fwd_enq = 0 ds_drop_new_flow = 0 ds_flow_evict = 0 ds_trap_original = 0 ds_no_memory = 0 ds_fragment_queue_fail = 0 ds_no_frag_entry = 0 ds_icmp_error = 0 ds_clone_fail = 0 ds_drop_pkts = 0 ] raw_drop_stats= [ ds_discard = 0 ds_pull = 0 ds_invalid_if = 0 ds_invalid_arp = 0 ds_trap_no_if = 0 ds_nowhere_to_go = 0 ds_flow_queue_limit_exceeded = 0 ds_flow_no_memory = 0 ds_flow_invalid_protocol = 0 ds_flow_nat_no_rflow = 0 ds_flow_action_drop = 0 ds_flow_action_invalid = 0 ds_flow_unusable = 0 ds_flow_table_full = 0 ds_interface_tx_discard = 0 ds_interface_drop = 0 ds_duplicated = 0 ds_push = 0 ds_ttl_exceeded = 0 ds_invalid_nh = 0 ds_invalid_label = 0 ds_invalid_protocol = 0 ds_interface_rx_discard = 0 ds_invalid_mcast_source = 0 ds_head_alloc_fail = 0 ds_pcow_fail = 0 ds_mcast_clone_fail = 0 ds_rewrite_fail = 0 ds_misc = 0 ds_invalid_packet = 0 ds_cksum_err = 0 ds_no_fmd = 0 ds_invalid_vnid = 0 ds_frag_err = 0 ds_invalid_source = 0 ds_mcast_df_bit = 0 ds_l2_no_route = 0 ds_vlan_fwd_tx = 0 ds_vlan_fwd_enq = 0 ds_drop_new_flow = 0 ds_flow_evict = 0 ds_trap_original = 0 ds_no_memory = 0 ds_fragment_queue_fail = 0 ds_no_frag_entry = 0 ds_icmp_error = 0 ds_clone_fail = 0 ds_drop_pkts = 0 ] total_in_bandwidth_utilization = 0 flow_rate= [ added_flows = 0 max_flow_adds_per_second = 0 min_flow_adds_per_second = 0 deleted_flows = 0 max_flow_deletes_per_second = 0 min_flow_deletes_per_second = 0 active_flows = 0 hold_flows = 0 ] phy_band_in_bps= [ [ _iter260->first = ens6 _iter260->second = 0, ] ] phy_band_out_bps= [ [ _iter261->first = ens6 _iter261->second = 33, ] ] raw_phy_if_stats= [ [ _iter262->first = ens6 [ in_pkts = 27 in_bytes = 2046 out_pkts = 196 out_bytes = 8428 ], ] ] raw_phy_if_drop_stats= [ [ _iter263->first = ens6 [ ds_discard = 0 ds_pull = 0 ds_invalid_if = 0 ds_invalid_arp = 0 ds_trap_no_if = 0 ds_nowhere_to_go = 0 ds_flow_queue_limit_exceeded = 0 ds_flow_no_memory = 0 ds_flow_invalid_protocol = 0 ds_flow_nat_no_rflow = 0 ds_flow_action_drop = 0 ds_flow_action_invalid = 0 ds_flow_unusable = 0 ds_flow_table_full = 0 ds_interface_tx_discard = 0 ds_interface_drop = 0 ds_duplicated = 0 ds_push = 0 ds_ttl_exceeded = 0 ds_invalid_nh = 0 ds_invalid_label = 0 ds_invalid_protocol = 0 ds_interface_rx_discard = 0 ds_invalid_mcast_source = 0 ds_head_alloc_fail = 0 ds_pcow_fail = 0 ds_mcast_clone_fail = 0 ds_rewrite_fail = 0 ds_misc = 0 ds_invalid_packet = 0 ds_cksum_err = 0 ds_no_fmd = 0 ds_invalid_vnid = 0 ds_frag_err = 0 ds_invalid_source = 0 ds_mcast_df_bit = 0 ds_l2_no_route = 0 ds_vlan_fwd_tx = 0 ds_vlan_fwd_enq = 0 ds_drop_new_flow = 0 ds_flow_evict = 0 ds_trap_original = 0 ds_no_memory = 0 ds_fragment_queue_fail = 0 ds_no_frag_entry = 0 ds_icmp_error = 0 ds_clone_fail = 0 ds_drop_pkts = 0 ], ] ] ] 2025-09-01 Mon 02:13:55:761.921 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848862754304, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: VrouterStats 2025-09-01 Mon 02:13:55:761.953 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848862754304, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:55:762.026 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848862754304, Pid 111888]: SANDESH: Send FAILED: 1756692835761306 [SYS_NOTICE]: VrouterControlStatsTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3992-1 raw_xmpp_stats= [ [ _iter306->first = 10.20.0.17 [ reconnects = 0 in_msgs = 0 out_msgs = 0 ], ] ] raw_ifmap_stats= [ [ _iter307->first = link_delete_parse_errors _iter307->second = 0, _iter307->first = link_update_parse_errors _iter307->second = 0, _iter307->first = node_delete_parse_errors _iter307->second = 0, _iter307->first = node_update_parse_errors _iter307->second = 0, ] ] raw_rt_table_size= [ [ _iter308->first = default-domain:default-project:ip-fabric:__default__ [ inet4_unicast = 4 inet4_multicast = 0 evpn = 0 bridge = 2 inet6_unicast = 2 ], _iter308->first = default-domain:default-project:ip-fabric:ip-fabric [ inet4_unicast = 2 inet4_multicast = 0 evpn = 1 bridge = 2 inet6_unicast = 0 ], ] ] ] 2025-09-01 Mon 02:13:55:762.047 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848862754304, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: VrouterControlStatsTrace 2025-09-01 Mon 02:13:56:817.707 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848866952704, Pid 111888]: Processing scm::EvIdleHoldTimerExpired in state Idle 2025-09-01 Mon 02:13:56:818.077 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848866952704, Pid 111888]: Connect : Start Connect timer 10.0.0.254:8086 2025-09-01 Mon 02:13:56:818.161 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848886536832, Pid 111888]: -(-1) OnSessionEvent TCP Connect Failed 2025-09-01 Mon 02:13:56:818.229 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848866952704, Pid 111888]: Processing scm::EvSandeshSend in state Connect 2025-09-01 Mon 02:13:56:818.240 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848886536832, Pid 111888]: -(-1):EvTcpConnectFail 2025-09-01 Mon 02:13:56:818.281 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848866952704, Pid 111888]: SANDESH: Send FAILED: 1756692836817859 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-3992-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.17 server_addrs= [ [ (*_iter6) = 10.20.0.17:5269, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.17 server_addrs= [ [ (*_iter6) = 10.20.0.17:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.254:8086, ] ] status = Initializing description = Idle : EvIdleHoldTimerExpired -> Connect ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.17, XMPP:dns-server:10.20.0.17, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:13:56:818.323 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848866952704, Pid 111888]: Wrong state: Connect for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:13:56:818.337 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848866952704, Pid 111888]: Processing scm::EvSandeshSend in state Connect 2025-09-01 Mon 02:13:56:818.571 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848866952704, Pid 111888]: SANDESH: Send FAILED: 1756692836818205 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3992-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Connect successful_connections = 0 pid = 111888 http_port = 8085 start_time = 1756692145743123 collector_name = collector_ip = 10.0.0.254:8086 collector_list= [ [ (*_iter40) = 10.0.0.254:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ComputeCpuStateTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 11 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 59 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 102 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 158 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterControlStatsTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 23 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 34 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 220 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 56 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 217, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 454, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:13:56:818.606 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848866952704, Pid 111888]: Wrong state: Connect for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:13:56:818.726 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848866952704, Pid 111888]: Processing scm::EvTcpConnectFail in state Connect 2025-09-01 Mon 02:13:56:818.739 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848866952704, Pid 111888]: Connect : EvTcpConnectFail 2025-09-01 Mon 02:13:56:818.804 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848866952704, Pid 111888]: TCP [SYS_ERR]: TcpSessionMessageLog: Session -(-1) > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:13:56:818.845 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848866952704, Pid 111888]: Idle 2025-09-01 Mon 02:13:56:818.933 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848866952704, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:56:818.962 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848866952704, Pid 111888]: SANDESH: Send FAILED: 1756692836818772 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-3992-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.17 server_addrs= [ [ (*_iter6) = 10.20.0.17:5269, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.17 server_addrs= [ [ (*_iter6) = 10.20.0.17:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.17, XMPP:dns-server:10.20.0.17, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:13:56:818.981 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848866952704, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:13:56:818.994 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848866952704, Pid 111888]: Processing scm::EvTcpDeleteSession in state Idle 2025-09-01 Mon 02:13:56:819.028 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848866952704, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:13:56:819.218 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848866952704, Pid 111888]: SANDESH: Send FAILED: 1756692836818917 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3992-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Idle successful_connections = 0 pid = 111888 http_port = 8085 start_time = 1756692145743123 collector_name = collector_ip = 10.0.0.254:8086 collector_list= [ [ (*_iter40) = 10.0.0.254:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ComputeCpuStateTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 11 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 60 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 103 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 159 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterControlStatsTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 23 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 34 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 220 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 56 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 218, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 456, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:13:56:819.244 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848866952704, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:14:06:079.483 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848886536832, Pid 111888]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.0.0.254:8085::10.0.0.254:57780 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 255 2025-09-01 Mon 02:14:12:611.785 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848879547904, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:12:611.889 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848879547904, Pid 111888]: SANDESH: Send FAILED: 1756692852611742 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.17 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:12:611.903 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848879547904, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:12:612.122 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848886536832, Pid 111888]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.17 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:14:12:612.178 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848879547904, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:12:612.211 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848879547904, Pid 111888]: SANDESH: Send FAILED: 1756692852612127 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.17 state = Connect event = Connect Event local_port = 40997 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:12:612.219 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848879547904, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:12:612.224 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848862754304, Pid 111888]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:14:12:612.468 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:12:612.539 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: SANDESH: Send FAILED: 1756692852612376 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.17 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:12:612.578 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:12:612.597 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:12:612.620 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: SANDESH: Send FAILED: 1756692852612411 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.17 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:12:612.634 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:18:963.123 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848886536832, Pid 111888]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.0.0.254:8085::10.0.0.254:33636 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 255 2025-09-01 Mon 02:14:22:138.434 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848879547904, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:22:138.524 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848879547904, Pid 111888]: SANDESH: Send FAILED: 1756692862138386 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.17 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:22:138.562 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848879547904, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:22:138.743 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848858555904, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:22:138.774 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848886536832, Pid 111888]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.17 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:14:22:138.825 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848858555904, Pid 111888]: SANDESH: Send FAILED: 1756692862138691 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.17 state = Connect event = Connect Event local_port = 39269 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:22:138.935 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848858555904, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:22:139.035 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848858555904, Pid 111888]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:14:22:139.133 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848862754304, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:22:139.226 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848862754304, Pid 111888]: SANDESH: Send FAILED: 1756692862139096 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.17 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:22:139.255 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848862754304, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:22:139.286 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848862754304, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:22:139.327 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848862754304, Pid 111888]: SANDESH: Send FAILED: 1756692862139112 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.17 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:22:139.351 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848862754304, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:25:757.225 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:25:757.357 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: SANDESH: Send FAILED: 1756692865757000 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3992-1:Compute:contrail-vrouter-agent:0 sm_queue_count = 0 sm_stats= [ ev_stats= [ [ [ event = scm::EvCollectorUpdate enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvIdleHoldTimerExpired enqueues = 27 dequeues = 27 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvSandeshSend enqueues = 466 dequeues = 466 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvStart enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpConnectFail enqueues = 28 dequeues = 28 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpDeleteSession enqueues = 28 dequeues = 28 enqueue_fails = 0 dequeue_fails = 0 ], [ event = enqueues = 551 dequeues = 551 enqueue_fails = 0 dequeue_fails = 0 ], ] ] state = Idle state_since = 1756692836818844 last_state = Idle last_event = scm::EvSandeshSend last_event_at = 1756692862139280 ] max_sm_queue_count = 6 ] 2025-09-01 Mon 02:14:25:757.376 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:14:25:757.397 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:25:757.832 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: SANDESH: Send FAILED: 1756692865757167 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3992-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Idle successful_connections = 0 pid = 111888 http_port = 8085 start_time = 1756692145743123 collector_name = collector_ip = 10.0.0.254:8086 collector_list= [ [ (*_iter40) = 10.0.0.254:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ComputeCpuStateTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 11 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 61 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 104 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 163 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterControlStatsTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 23 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 34 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 228 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 58 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 224, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 466, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:14:25:757.874 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:14:25:761.566 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848866952704, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:25:761.688 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848866952704, Pid 111888]: SANDESH: Send FAILED: 1756692865761504 [SYS_NOTICE]: VrouterStats: data= [ name = cn-jenkins-deploy-platform-ansible-os-3992-1 cpu_info= [ num_cpu = 8 sys_mem_info= [ total = 32 used = 0 free = 0 buffers = 0 cached = 510398136 node_type = ] meminfo= [ virt = 32561 peakvirt = 0 res = 0 ] cpuload= [ one_min_avg = 0.135 five_min_avg = 0.18375 fifteen_min_avg = 0.20875 ] cpu_share = 0.28125 ] ] 2025-09-01 Mon 02:14:25:761.706 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848866952704, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: VrouterStats 2025-09-01 Mon 02:14:25:761.720 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848866952704, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:25:761.743 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848866952704, Pid 111888]: SANDESH: Send FAILED: 1756692865761522 [SYS_NOTICE]: ComputeCpuStateTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3992-1 cpu_info= [ [ [ mem_virt = 32561 cpu_share = 0.28125 used_sys_mem = 3784569160 one_min_cpuload = 0.135 mem_res = 0 ], ] ] ] 2025-09-01 Mon 02:14:25:761.754 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848866952704, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: ComputeCpuStateTrace 2025-09-01 Mon 02:14:25:761.765 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848866952704, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:25:761.983 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848866952704, Pid 111888]: SANDESH: Send FAILED: 1756692865761573 [SYS_NOTICE]: VrouterStats: data= [ name = cn-jenkins-deploy-platform-ansible-os-3992-1 in_tpkts = 199 in_bytes = 8554 out_tpkts = 219 out_bytes = 10110 raw_vhost_stats= [ name = vhost0 in_pkts = 199 in_bytes = 8554 out_pkts = 219 out_bytes = 10110 speed = -1 duplexity = -1 ] raw_vhost_drop_stats= [ ds_discard = 0 ds_pull = 0 ds_invalid_if = 0 ds_invalid_arp = 0 ds_trap_no_if = 0 ds_nowhere_to_go = 0 ds_flow_queue_limit_exceeded = 0 ds_flow_no_memory = 0 ds_flow_invalid_protocol = 0 ds_flow_nat_no_rflow = 0 ds_flow_action_drop = 0 ds_flow_action_invalid = 0 ds_flow_unusable = 0 ds_flow_table_full = 0 ds_interface_tx_discard = 0 ds_interface_drop = 0 ds_duplicated = 0 ds_push = 0 ds_ttl_exceeded = 0 ds_invalid_nh = 0 ds_invalid_label = 0 ds_invalid_protocol = 0 ds_interface_rx_discard = 0 ds_invalid_mcast_source = 0 ds_head_alloc_fail = 0 ds_pcow_fail = 0 ds_mcast_clone_fail = 0 ds_rewrite_fail = 0 ds_misc = 0 ds_invalid_packet = 0 ds_cksum_err = 0 ds_no_fmd = 0 ds_invalid_vnid = 0 ds_frag_err = 0 ds_invalid_source = 0 ds_mcast_df_bit = 0 ds_l2_no_route = 0 ds_vlan_fwd_tx = 0 ds_vlan_fwd_enq = 0 ds_drop_new_flow = 0 ds_flow_evict = 0 ds_trap_original = 0 ds_no_memory = 0 ds_fragment_queue_fail = 0 ds_no_frag_entry = 0 ds_icmp_error = 0 ds_clone_fail = 0 ds_drop_pkts = 0 ] raw_drop_stats= [ ds_discard = 0 ds_pull = 0 ds_invalid_if = 0 ds_invalid_arp = 0 ds_trap_no_if = 0 ds_nowhere_to_go = 0 ds_flow_queue_limit_exceeded = 0 ds_flow_no_memory = 0 ds_flow_invalid_protocol = 0 ds_flow_nat_no_rflow = 0 ds_flow_action_drop = 0 ds_flow_action_invalid = 0 ds_flow_unusable = 0 ds_flow_table_full = 0 ds_interface_tx_discard = 0 ds_interface_drop = 0 ds_duplicated = 0 ds_push = 0 ds_ttl_exceeded = 0 ds_invalid_nh = 0 ds_invalid_label = 0 ds_invalid_protocol = 0 ds_interface_rx_discard = 0 ds_invalid_mcast_source = 0 ds_head_alloc_fail = 0 ds_pcow_fail = 0 ds_mcast_clone_fail = 0 ds_rewrite_fail = 0 ds_misc = 0 ds_invalid_packet = 0 ds_cksum_err = 0 ds_no_fmd = 0 ds_invalid_vnid = 0 ds_frag_err = 0 ds_invalid_source = 0 ds_mcast_df_bit = 0 ds_l2_no_route = 0 ds_vlan_fwd_tx = 0 ds_vlan_fwd_enq = 0 ds_drop_new_flow = 0 ds_flow_evict = 0 ds_trap_original = 0 ds_no_memory = 0 ds_fragment_queue_fail = 0 ds_no_frag_entry = 0 ds_icmp_error = 0 ds_clone_fail = 0 ds_drop_pkts = 0 ] flow_rate= [ added_flows = 0 max_flow_adds_per_second = 0 min_flow_adds_per_second = 0 deleted_flows = 0 max_flow_deletes_per_second = 0 min_flow_deletes_per_second = 0 active_flows = 0 hold_flows = 0 ] raw_phy_if_stats= [ [ _iter262->first = ens6 [ in_pkts = 27 in_bytes = 2046 out_pkts = 199 out_bytes = 8554 ], ] ] raw_phy_if_drop_stats= [ [ _iter263->first = ens6 [ ds_discard = 0 ds_pull = 0 ds_invalid_if = 0 ds_invalid_arp = 0 ds_trap_no_if = 0 ds_nowhere_to_go = 0 ds_flow_queue_limit_exceeded = 0 ds_flow_no_memory = 0 ds_flow_invalid_protocol = 0 ds_flow_nat_no_rflow = 0 ds_flow_action_drop = 0 ds_flow_action_invalid = 0 ds_flow_unusable = 0 ds_flow_table_full = 0 ds_interface_tx_discard = 0 ds_interface_drop = 0 ds_duplicated = 0 ds_push = 0 ds_ttl_exceeded = 0 ds_invalid_nh = 0 ds_invalid_label = 0 ds_invalid_protocol = 0 ds_interface_rx_discard = 0 ds_invalid_mcast_source = 0 ds_head_alloc_fail = 0 ds_pcow_fail = 0 ds_mcast_clone_fail = 0 ds_rewrite_fail = 0 ds_misc = 0 ds_invalid_packet = 0 ds_cksum_err = 0 ds_no_fmd = 0 ds_invalid_vnid = 0 ds_frag_err = 0 ds_invalid_source = 0 ds_mcast_df_bit = 0 ds_l2_no_route = 0 ds_vlan_fwd_tx = 0 ds_vlan_fwd_enq = 0 ds_drop_new_flow = 0 ds_flow_evict = 0 ds_trap_original = 0 ds_no_memory = 0 ds_fragment_queue_fail = 0 ds_no_frag_entry = 0 ds_icmp_error = 0 ds_clone_fail = 0 ds_drop_pkts = 0 ], ] ] ] 2025-09-01 Mon 02:14:25:762.018 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848866952704, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: VrouterStats 2025-09-01 Mon 02:14:25:762.031 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848866952704, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:25:762.073 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848866952704, Pid 111888]: SANDESH: Send FAILED: 1756692865761644 [SYS_NOTICE]: VrouterControlStatsTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3992-1 raw_xmpp_stats= [ [ _iter306->first = 10.20.0.17 [ reconnects = 0 in_msgs = 0 out_msgs = 0 ], ] ] raw_ifmap_stats= [ [ _iter307->first = link_delete_parse_errors _iter307->second = 0, _iter307->first = link_update_parse_errors _iter307->second = 0, _iter307->first = node_delete_parse_errors _iter307->second = 0, _iter307->first = node_update_parse_errors _iter307->second = 0, ] ] raw_rt_table_size= [ [ _iter308->first = default-domain:default-project:ip-fabric:__default__ [ inet4_unicast = 4 inet4_multicast = 0 evpn = 0 bridge = 2 inet6_unicast = 2 ], _iter308->first = default-domain:default-project:ip-fabric:ip-fabric [ inet4_unicast = 2 inet4_multicast = 0 evpn = 1 bridge = 2 inet6_unicast = 0 ], ] ] ] 2025-09-01 Mon 02:14:25:762.083 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848866952704, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: VrouterControlStatsTrace 2025-09-01 Mon 02:14:26:819.088 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848879547904, Pid 111888]: Processing scm::EvIdleHoldTimerExpired in state Idle 2025-09-01 Mon 02:14:26:819.617 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848879547904, Pid 111888]: Connect : Start Connect timer 10.0.0.254:8086 2025-09-01 Mon 02:14:26:819.660 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848886536832, Pid 111888]: -(-1) OnSessionEvent TCP Connect Failed 2025-09-01 Mon 02:14:26:819.728 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848886536832, Pid 111888]: -(-1):EvTcpConnectFail 2025-09-01 Mon 02:14:26:819.871 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848879547904, Pid 111888]: Processing scm::EvSandeshSend in state Connect 2025-09-01 Mon 02:14:26:819.943 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848879547904, Pid 111888]: SANDESH: Send FAILED: 1756692866819258 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-3992-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.17 server_addrs= [ [ (*_iter6) = 10.20.0.17:5269, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.17 server_addrs= [ [ (*_iter6) = 10.20.0.17:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.254:8086, ] ] status = Initializing description = Idle : EvIdleHoldTimerExpired -> Connect ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.17, XMPP:dns-server:10.20.0.17, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:14:26:819.989 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848879547904, Pid 111888]: Wrong state: Connect for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:14:26:820.017 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848879547904, Pid 111888]: Processing scm::EvTcpConnectFail in state Connect 2025-09-01 Mon 02:14:26:820.037 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848879547904, Pid 111888]: Connect : EvTcpConnectFail 2025-09-01 Mon 02:14:26:820.144 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848879547904, Pid 111888]: TCP [SYS_ERR]: TcpSessionMessageLog: Session -(-1) > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:14:26:820.210 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848879547904, Pid 111888]: Idle 2025-09-01 Mon 02:14:26:820.353 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848879547904, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:26:820.963 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848879547904, Pid 111888]: SANDESH: Send FAILED: 1756692866819799 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3992-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Connect successful_connections = 0 pid = 111888 http_port = 8085 start_time = 1756692145743123 collector_name = collector_ip = 10.0.0.254:8086 collector_list= [ [ (*_iter40) = 10.0.0.254:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ComputeCpuStateTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 12 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 61 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 106 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 163 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterControlStatsTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 24 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 36 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 228 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 58 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 224, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 472, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:14:26:821.033 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848879547904, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:14:26:821.064 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848879547904, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:26:821.116 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848879547904, Pid 111888]: SANDESH: Send FAILED: 1756692866820091 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-3992-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.17 server_addrs= [ [ (*_iter6) = 10.20.0.17:5269, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.17 server_addrs= [ [ (*_iter6) = 10.20.0.17:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.17, XMPP:dns-server:10.20.0.17, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:14:26:821.136 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848879547904, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:14:26:821.167 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848879547904, Pid 111888]: Processing scm::EvTcpDeleteSession in state Idle 2025-09-01 Mon 02:14:26:821.235 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848879547904, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:26:821.624 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848879547904, Pid 111888]: SANDESH: Send FAILED: 1756692866820323 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3992-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Idle successful_connections = 0 pid = 111888 http_port = 8085 start_time = 1756692145743123 collector_name = collector_ip = 10.0.0.254:8086 collector_list= [ [ (*_iter40) = 10.0.0.254:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ComputeCpuStateTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 12 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 62 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 106 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 164 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterControlStatsTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 24 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 36 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 228 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 58 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 225, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 473, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:14:26:821.723 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848879547904, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:14:32:007.630 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848886536832, Pid 111888]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.0.0.254:8085::10.0.0.254:50518 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 255 2025-09-01 Mon 02:14:35:604.177 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848886536832, Pid 111888]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.20.0.17:8085::10.20.0.17:56246 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 255 2025-09-01 Mon 02:14:35:628.915 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848886536832, Pid 111888]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.20.0.17:8085::10.20.0.17:56248 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 255 2025-09-01 Mon 02:14:40:152.821 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848862754304, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:40:152.989 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848862754304, Pid 111888]: SANDESH: Send FAILED: 1756692880152740 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.17 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:40:153.019 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848862754304, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:40:153.119 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:40:153.162 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: SANDESH: Send FAILED: 1756692880153057 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.17 state = Connect event = Connect Event local_port = 40515 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:40:153.198 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:40:153.161 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848886536832, Pid 111888]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.17 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:14:40:153.535 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848866952704, Pid 111888]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:14:40:153.947 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:40:154.002 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: SANDESH: Send FAILED: 1756692880153823 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.17 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:40:154.017 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:40:154.030 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:40:154.045 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: SANDESH: Send FAILED: 1756692880153887 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.17 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:40:154.054 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:44:892.132 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848886536832, Pid 111888]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.0.0.254:8085::10.0.0.254:35430 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 255 2025-09-01 Mon 02:14:53:729.675 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848858555904, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:53:729.739 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848858555904, Pid 111888]: SANDESH: Send FAILED: 1756692893729469 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.17 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:53:729.751 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848858555904, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:53:729.954 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:53:729.987 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: SANDESH: Send FAILED: 1756692893729939 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.17 state = Connect event = Connect Event local_port = 34037 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:53:729.998 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:53:730.020 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848886536832, Pid 111888]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.17 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:14:53:730.185 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848862754304, Pid 111888]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:14:53:730.249 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848854357504, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:53:730.334 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848854357504, Pid 111888]: SANDESH: Send FAILED: 1756692893730219 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.17 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:53:730.381 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848854357504, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:53:730.417 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848854357504, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:53:730.447 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848854357504, Pid 111888]: SANDESH: Send FAILED: 1756692893730226 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.17 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:14:53:730.487 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848854357504, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:14:55:757.894 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:55:758.100 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: SANDESH: Send FAILED: 1756692895757427 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3992-1:Compute:contrail-vrouter-agent:0 sm_queue_count = 0 sm_stats= [ ev_stats= [ [ [ event = scm::EvCollectorUpdate enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvIdleHoldTimerExpired enqueues = 28 dequeues = 28 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvSandeshSend enqueues = 484 dequeues = 484 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvStart enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpConnectFail enqueues = 29 dequeues = 29 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpDeleteSession enqueues = 29 dequeues = 29 enqueue_fails = 0 dequeue_fails = 0 ], [ event = enqueues = 572 dequeues = 572 enqueue_fails = 0 dequeue_fails = 0 ], ] ] state = Idle state_since = 1756692866820208 last_state = Idle last_event = scm::EvSandeshSend last_event_at = 1756692893730413 ] max_sm_queue_count = 6 ] 2025-09-01 Mon 02:14:55:758.137 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:14:55:758.164 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:55:758.602 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: SANDESH: Send FAILED: 1756692895757765 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3992-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Idle successful_connections = 0 pid = 111888 http_port = 8085 start_time = 1756692145743123 collector_name = collector_ip = 10.0.0.254:8086 collector_list= [ [ (*_iter40) = 10.0.0.254:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ComputeCpuStateTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 12 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 63 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 108 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 170 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterControlStatsTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 24 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 36 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 236 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 60 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 233, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 484, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:14:55:758.669 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:14:55:762.063 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848862754304, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:55:762.462 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848862754304, Pid 111888]: SANDESH: Send FAILED: 1756692895761994 [SYS_NOTICE]: VrouterStats: data= [ name = cn-jenkins-deploy-platform-ansible-os-3992-1 in_tpkts = 202 in_bytes = 8680 out_tpkts = 222 out_bytes = 10236 raw_vhost_stats= [ name = vhost0 in_pkts = 202 in_bytes = 8680 out_pkts = 222 out_bytes = 10236 speed = -1 duplexity = -1 ] raw_vhost_drop_stats= [ ds_discard = 0 ds_pull = 0 ds_invalid_if = 0 ds_invalid_arp = 0 ds_trap_no_if = 0 ds_nowhere_to_go = 0 ds_flow_queue_limit_exceeded = 0 ds_flow_no_memory = 0 ds_flow_invalid_protocol = 0 ds_flow_nat_no_rflow = 0 ds_flow_action_drop = 0 ds_flow_action_invalid = 0 ds_flow_unusable = 0 ds_flow_table_full = 0 ds_interface_tx_discard = 0 ds_interface_drop = 0 ds_duplicated = 0 ds_push = 0 ds_ttl_exceeded = 0 ds_invalid_nh = 0 ds_invalid_label = 0 ds_invalid_protocol = 0 ds_interface_rx_discard = 0 ds_invalid_mcast_source = 0 ds_head_alloc_fail = 0 ds_pcow_fail = 0 ds_mcast_clone_fail = 0 ds_rewrite_fail = 0 ds_misc = 0 ds_invalid_packet = 0 ds_cksum_err = 0 ds_no_fmd = 0 ds_invalid_vnid = 0 ds_frag_err = 0 ds_invalid_source = 0 ds_mcast_df_bit = 0 ds_l2_no_route = 0 ds_vlan_fwd_tx = 0 ds_vlan_fwd_enq = 0 ds_drop_new_flow = 0 ds_flow_evict = 0 ds_trap_original = 0 ds_no_memory = 0 ds_fragment_queue_fail = 0 ds_no_frag_entry = 0 ds_icmp_error = 0 ds_clone_fail = 0 ds_drop_pkts = 0 ] raw_drop_stats= [ ds_discard = 0 ds_pull = 0 ds_invalid_if = 0 ds_invalid_arp = 0 ds_trap_no_if = 0 ds_nowhere_to_go = 0 ds_flow_queue_limit_exceeded = 0 ds_flow_no_memory = 0 ds_flow_invalid_protocol = 0 ds_flow_nat_no_rflow = 0 ds_flow_action_drop = 0 ds_flow_action_invalid = 0 ds_flow_unusable = 0 ds_flow_table_full = 0 ds_interface_tx_discard = 0 ds_interface_drop = 0 ds_duplicated = 0 ds_push = 0 ds_ttl_exceeded = 0 ds_invalid_nh = 0 ds_invalid_label = 0 ds_invalid_protocol = 0 ds_interface_rx_discard = 0 ds_invalid_mcast_source = 0 ds_head_alloc_fail = 0 ds_pcow_fail = 0 ds_mcast_clone_fail = 0 ds_rewrite_fail = 0 ds_misc = 0 ds_invalid_packet = 0 ds_cksum_err = 0 ds_no_fmd = 0 ds_invalid_vnid = 0 ds_frag_err = 0 ds_invalid_source = 0 ds_mcast_df_bit = 0 ds_l2_no_route = 0 ds_vlan_fwd_tx = 0 ds_vlan_fwd_enq = 0 ds_drop_new_flow = 0 ds_flow_evict = 0 ds_trap_original = 0 ds_no_memory = 0 ds_fragment_queue_fail = 0 ds_no_frag_entry = 0 ds_icmp_error = 0 ds_clone_fail = 0 ds_drop_pkts = 0 ] flow_rate= [ added_flows = 0 max_flow_adds_per_second = 0 min_flow_adds_per_second = 0 deleted_flows = 0 max_flow_deletes_per_second = 0 min_flow_deletes_per_second = 0 active_flows = 0 hold_flows = 0 ] phy_band_in_bps= [ [ _iter260->first = ens6 _iter260->second = 0, ] ] phy_band_out_bps= [ [ _iter261->first = ens6 _iter261->second = 33, ] ] raw_phy_if_stats= [ [ _iter262->first = ens6 [ in_pkts = 27 in_bytes = 2046 out_pkts = 202 out_bytes = 8680 ], ] ] raw_phy_if_drop_stats= [ [ _iter263->first = ens6 [ ds_discard = 0 ds_pull = 0 ds_invalid_if = 0 ds_invalid_arp = 0 ds_trap_no_if = 0 ds_nowhere_to_go = 0 ds_flow_queue_limit_exceeded = 0 ds_flow_no_memory = 0 ds_flow_invalid_protocol = 0 ds_flow_nat_no_rflow = 0 ds_flow_action_drop = 0 ds_flow_action_invalid = 0 ds_flow_unusable = 0 ds_flow_table_full = 0 ds_interface_tx_discard = 0 ds_interface_drop = 0 ds_duplicated = 0 ds_push = 0 ds_ttl_exceeded = 0 ds_invalid_nh = 0 ds_invalid_label = 0 ds_invalid_protocol = 0 ds_interface_rx_discard = 0 ds_invalid_mcast_source = 0 ds_head_alloc_fail = 0 ds_pcow_fail = 0 ds_mcast_clone_fail = 0 ds_rewrite_fail = 0 ds_misc = 0 ds_invalid_packet = 0 ds_cksum_err = 0 ds_no_fmd = 0 ds_invalid_vnid = 0 ds_frag_err = 0 ds_invalid_source = 0 ds_mcast_df_bit = 0 ds_l2_no_route = 0 ds_vlan_fwd_tx = 0 ds_vlan_fwd_enq = 0 ds_drop_new_flow = 0 ds_flow_evict = 0 ds_trap_original = 0 ds_no_memory = 0 ds_fragment_queue_fail = 0 ds_no_frag_entry = 0 ds_icmp_error = 0 ds_clone_fail = 0 ds_drop_pkts = 0 ], ] ] ] 2025-09-01 Mon 02:14:55:762.530 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848862754304, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: VrouterStats 2025-09-01 Mon 02:14:55:762.578 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848862754304, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:55:762.676 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848862754304, Pid 111888]: SANDESH: Send FAILED: 1756692895762082 [SYS_NOTICE]: VrouterControlStatsTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3992-1 raw_xmpp_stats= [ [ _iter306->first = 10.20.0.17 [ reconnects = 0 in_msgs = 0 out_msgs = 0 ], ] ] raw_ifmap_stats= [ [ _iter307->first = link_delete_parse_errors _iter307->second = 0, _iter307->first = link_update_parse_errors _iter307->second = 0, _iter307->first = node_delete_parse_errors _iter307->second = 0, _iter307->first = node_update_parse_errors _iter307->second = 0, ] ] raw_rt_table_size= [ [ _iter308->first = default-domain:default-project:ip-fabric:__default__ [ inet4_unicast = 4 inet4_multicast = 0 evpn = 0 bridge = 2 inet6_unicast = 2 ], _iter308->first = default-domain:default-project:ip-fabric:ip-fabric [ inet4_unicast = 2 inet4_multicast = 0 evpn = 1 bridge = 2 inet6_unicast = 0 ], ] ] ] 2025-09-01 Mon 02:14:55:762.697 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848862754304, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: VrouterControlStatsTrace 2025-09-01 Mon 02:14:56:820.459 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848866952704, Pid 111888]: Processing scm::EvIdleHoldTimerExpired in state Idle 2025-09-01 Mon 02:14:56:821.081 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848866952704, Pid 111888]: Connect : Start Connect timer 10.0.0.254:8086 2025-09-01 Mon 02:14:56:821.151 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848886536832, Pid 111888]: -(-1) OnSessionEvent TCP Connect Failed 2025-09-01 Mon 02:14:56:821.208 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848886536832, Pid 111888]: -(-1):EvTcpConnectFail 2025-09-01 Mon 02:14:56:821.385 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848866952704, Pid 111888]: Processing scm::EvSandeshSend in state Connect 2025-09-01 Mon 02:14:56:821.480 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848866952704, Pid 111888]: SANDESH: Send FAILED: 1756692896820695 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-3992-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.17 server_addrs= [ [ (*_iter6) = 10.20.0.17:5269, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.17 server_addrs= [ [ (*_iter6) = 10.20.0.17:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.254:8086, ] ] status = Initializing description = Idle : EvIdleHoldTimerExpired -> Connect ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.17, XMPP:dns-server:10.20.0.17, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:14:56:821.533 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848866952704, Pid 111888]: Wrong state: Connect for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:14:56:821.585 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848866952704, Pid 111888]: Processing scm::EvTcpConnectFail in state Connect 2025-09-01 Mon 02:14:56:821.606 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848866952704, Pid 111888]: Connect : EvTcpConnectFail 2025-09-01 Mon 02:14:56:821.786 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848866952704, Pid 111888]: TCP [SYS_ERR]: TcpSessionMessageLog: Session -(-1) > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:14:56:821.864 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848866952704, Pid 111888]: Idle 2025-09-01 Mon 02:14:56:822.056 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848866952704, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:56:822.663 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848866952704, Pid 111888]: SANDESH: Send FAILED: 1756692896821315 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3992-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Connect successful_connections = 0 pid = 111888 http_port = 8085 start_time = 1756692145743123 collector_name = collector_ip = 10.0.0.254:8086 collector_list= [ [ (*_iter40) = 10.0.0.254:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ComputeCpuStateTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 12 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 63 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 110 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 170 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterControlStatsTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 25 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 37 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 236 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 60 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 233, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 488, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:14:56:822.723 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848866952704, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:14:56:822.753 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848866952704, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:56:822.837 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848866952704, Pid 111888]: SANDESH: Send FAILED: 1756692896821731 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-3992-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.17 server_addrs= [ [ (*_iter6) = 10.20.0.17:5269, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.17 server_addrs= [ [ (*_iter6) = 10.20.0.17:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.17, XMPP:dns-server:10.20.0.17, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:14:56:822.857 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848866952704, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:14:56:822.879 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848866952704, Pid 111888]: Processing scm::EvTcpDeleteSession in state Idle 2025-09-01 Mon 02:14:56:822.940 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848866952704, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:14:56:823.306 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848866952704, Pid 111888]: SANDESH: Send FAILED: 1756692896821982 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3992-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Idle successful_connections = 0 pid = 111888 http_port = 8085 start_time = 1756692145743123 collector_name = collector_ip = 10.0.0.254:8086 collector_list= [ [ (*_iter40) = 10.0.0.254:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ComputeCpuStateTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 12 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 64 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 110 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 171 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterControlStatsTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 25 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 37 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 236 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 60 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 234, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 489, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:14:56:823.353 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848866952704, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:14:57:840.648 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848886536832, Pid 111888]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.0.0.254:8085::10.0.0.254:42074 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 255 2025-09-01 Mon 02:15:10:664.299 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848879547904, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:10:664.397 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848879547904, Pid 111888]: SANDESH: Send FAILED: 1756692910664131 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.17 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:10:664.432 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848879547904, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:10:664.628 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848886536832, Pid 111888]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.17 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:15:10:664.664 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:10:664.708 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: SANDESH: Send FAILED: 1756692910664601 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.17 state = Connect event = Connect Event local_port = 38667 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:10:664.742 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:10:664.744 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848854357504, Pid 111888]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:15:10:664.932 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848866952704, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:10:665.019 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848866952704, Pid 111888]: SANDESH: Send FAILED: 1756692910664885 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.17 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:10:665.045 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848866952704, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:10:665.071 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848866952704, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:10:665.104 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848866952704, Pid 111888]: SANDESH: Send FAILED: 1756692910664897 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.17 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:10:665.122 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848866952704, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:10:894.542 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848886536832, Pid 111888]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.0.0.254:8085::10.0.0.254:33836 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 255 2025-09-01 Mon 02:15:23:748.242 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848886536832, Pid 111888]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.0.0.254:8085::10.0.0.254:54526 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 255 2025-09-01 Mon 02:15:24:780.613 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848854357504, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:24:780.739 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848854357504, Pid 111888]: SANDESH: Send FAILED: 1756692924780516 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.17 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:24:780.790 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848854357504, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:24:781.034 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848862754304, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:24:781.090 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848886536832, Pid 111888]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.17 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:15:24:781.168 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848862754304, Pid 111888]: SANDESH: Send FAILED: 1756692924780960 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.17 state = Connect event = Connect Event local_port = 33239 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:24:781.248 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848862754304, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:24:781.336 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:15:24:781.595 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848879547904, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:24:781.732 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848879547904, Pid 111888]: SANDESH: Send FAILED: 1756692924781451 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.17 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:24:781.777 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848879547904, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:24:781.812 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848879547904, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:24:781.852 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848879547904, Pid 111888]: SANDESH: Send FAILED: 1756692924781513 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.17 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:24:781.876 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848879547904, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:25:758.366 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848866952704, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:25:758.543 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848866952704, Pid 111888]: SANDESH: Send FAILED: 1756692925758038 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3992-1:Compute:contrail-vrouter-agent:0 sm_queue_count = 0 sm_stats= [ ev_stats= [ [ [ event = scm::EvCollectorUpdate enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvIdleHoldTimerExpired enqueues = 29 dequeues = 29 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvSandeshSend enqueues = 500 dequeues = 500 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvStart enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpConnectFail enqueues = 30 dequeues = 30 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpDeleteSession enqueues = 30 dequeues = 30 enqueue_fails = 0 dequeue_fails = 0 ], [ event = enqueues = 591 dequeues = 591 enqueue_fails = 0 dequeue_fails = 0 ], ] ] state = Idle state_since = 1756692896821861 last_state = Idle last_event = scm::EvSandeshSend last_event_at = 1756692924781806 ] max_sm_queue_count = 6 ] 2025-09-01 Mon 02:15:25:758.631 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848866952704, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:15:25:758.658 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848866952704, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:25:759.110 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848866952704, Pid 111888]: SANDESH: Send FAILED: 1756692925758241 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3992-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Idle successful_connections = 0 pid = 111888 http_port = 8085 start_time = 1756692145743123 collector_name = collector_ip = 10.0.0.254:8086 collector_list= [ [ (*_iter40) = 10.0.0.254:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ComputeCpuStateTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 12 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 65 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 112 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 176 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterControlStatsTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 25 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 37 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 244 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 62 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 241, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 500, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:15:25:759.162 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848866952704, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:15:25:762.332 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848854357504, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:25:762.482 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848854357504, Pid 111888]: SANDESH: Send FAILED: 1756692925762286 [SYS_NOTICE]: VrouterStats: data= [ name = cn-jenkins-deploy-platform-ansible-os-3992-1 cpu_info= [ num_cpu = 8 sys_mem_info= [ total = 32 used = 0 free = 0 buffers = 0 cached = 510398136 node_type = ] meminfo= [ virt = 32561 peakvirt = 0 res = 0 ] cpuload= [ one_min_avg = 0.1775 five_min_avg = 0.19 fifteen_min_avg = 0.20875 ] cpu_share = 0.44375 ] ] 2025-09-01 Mon 02:15:25:762.511 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848854357504, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: VrouterStats 2025-09-01 Mon 02:15:25:762.538 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848854357504, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:25:762.631 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848854357504, Pid 111888]: SANDESH: Send FAILED: 1756692925762302 [SYS_NOTICE]: ComputeCpuStateTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3992-1 cpu_info= [ [ [ mem_virt = 32561 cpu_share = 0.44375 used_sys_mem = 3784569160 one_min_cpuload = 0.1775 mem_res = 0 ], ] ] ] 2025-09-01 Mon 02:15:25:762.653 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848854357504, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: ComputeCpuStateTrace 2025-09-01 Mon 02:15:25:762.676 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848854357504, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:25:763.047 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848854357504, Pid 111888]: SANDESH: Send FAILED: 1756692925762333 [SYS_NOTICE]: VrouterStats: data= [ name = cn-jenkins-deploy-platform-ansible-os-3992-1 in_tpkts = 205 in_bytes = 8806 out_tpkts = 225 out_bytes = 10362 raw_vhost_stats= [ name = vhost0 in_pkts = 205 in_bytes = 8806 out_pkts = 225 out_bytes = 10362 speed = -1 duplexity = -1 ] raw_vhost_drop_stats= [ ds_discard = 0 ds_pull = 0 ds_invalid_if = 0 ds_invalid_arp = 0 ds_trap_no_if = 0 ds_nowhere_to_go = 0 ds_flow_queue_limit_exceeded = 0 ds_flow_no_memory = 0 ds_flow_invalid_protocol = 0 ds_flow_nat_no_rflow = 0 ds_flow_action_drop = 0 ds_flow_action_invalid = 0 ds_flow_unusable = 0 ds_flow_table_full = 0 ds_interface_tx_discard = 0 ds_interface_drop = 0 ds_duplicated = 0 ds_push = 0 ds_ttl_exceeded = 0 ds_invalid_nh = 0 ds_invalid_label = 0 ds_invalid_protocol = 0 ds_interface_rx_discard = 0 ds_invalid_mcast_source = 0 ds_head_alloc_fail = 0 ds_pcow_fail = 0 ds_mcast_clone_fail = 0 ds_rewrite_fail = 0 ds_misc = 0 ds_invalid_packet = 0 ds_cksum_err = 0 ds_no_fmd = 0 ds_invalid_vnid = 0 ds_frag_err = 0 ds_invalid_source = 0 ds_mcast_df_bit = 0 ds_l2_no_route = 0 ds_vlan_fwd_tx = 0 ds_vlan_fwd_enq = 0 ds_drop_new_flow = 0 ds_flow_evict = 0 ds_trap_original = 0 ds_no_memory = 0 ds_fragment_queue_fail = 0 ds_no_frag_entry = 0 ds_icmp_error = 0 ds_clone_fail = 0 ds_drop_pkts = 0 ] raw_drop_stats= [ ds_discard = 0 ds_pull = 0 ds_invalid_if = 0 ds_invalid_arp = 0 ds_trap_no_if = 0 ds_nowhere_to_go = 0 ds_flow_queue_limit_exceeded = 0 ds_flow_no_memory = 0 ds_flow_invalid_protocol = 0 ds_flow_nat_no_rflow = 0 ds_flow_action_drop = 0 ds_flow_action_invalid = 0 ds_flow_unusable = 0 ds_flow_table_full = 0 ds_interface_tx_discard = 0 ds_interface_drop = 0 ds_duplicated = 0 ds_push = 0 ds_ttl_exceeded = 0 ds_invalid_nh = 0 ds_invalid_label = 0 ds_invalid_protocol = 0 ds_interface_rx_discard = 0 ds_invalid_mcast_source = 0 ds_head_alloc_fail = 0 ds_pcow_fail = 0 ds_mcast_clone_fail = 0 ds_rewrite_fail = 0 ds_misc = 0 ds_invalid_packet = 0 ds_cksum_err = 0 ds_no_fmd = 0 ds_invalid_vnid = 0 ds_frag_err = 0 ds_invalid_source = 0 ds_mcast_df_bit = 0 ds_l2_no_route = 0 ds_vlan_fwd_tx = 0 ds_vlan_fwd_enq = 0 ds_drop_new_flow = 0 ds_flow_evict = 0 ds_trap_original = 0 ds_no_memory = 0 ds_fragment_queue_fail = 0 ds_no_frag_entry = 0 ds_icmp_error = 0 ds_clone_fail = 0 ds_drop_pkts = 0 ] flow_rate= [ added_flows = 0 max_flow_adds_per_second = 0 min_flow_adds_per_second = 0 deleted_flows = 0 max_flow_deletes_per_second = 0 min_flow_deletes_per_second = 0 active_flows = 0 hold_flows = 0 ] raw_phy_if_stats= [ [ _iter262->first = ens6 [ in_pkts = 27 in_bytes = 2046 out_pkts = 205 out_bytes = 8806 ], ] ] raw_phy_if_drop_stats= [ [ _iter263->first = ens6 [ ds_discard = 0 ds_pull = 0 ds_invalid_if = 0 ds_invalid_arp = 0 ds_trap_no_if = 0 ds_nowhere_to_go = 0 ds_flow_queue_limit_exceeded = 0 ds_flow_no_memory = 0 ds_flow_invalid_protocol = 0 ds_flow_nat_no_rflow = 0 ds_flow_action_drop = 0 ds_flow_action_invalid = 0 ds_flow_unusable = 0 ds_flow_table_full = 0 ds_interface_tx_discard = 0 ds_interface_drop = 0 ds_duplicated = 0 ds_push = 0 ds_ttl_exceeded = 0 ds_invalid_nh = 0 ds_invalid_label = 0 ds_invalid_protocol = 0 ds_interface_rx_discard = 0 ds_invalid_mcast_source = 0 ds_head_alloc_fail = 0 ds_pcow_fail = 0 ds_mcast_clone_fail = 0 ds_rewrite_fail = 0 ds_misc = 0 ds_invalid_packet = 0 ds_cksum_err = 0 ds_no_fmd = 0 ds_invalid_vnid = 0 ds_frag_err = 0 ds_invalid_source = 0 ds_mcast_df_bit = 0 ds_l2_no_route = 0 ds_vlan_fwd_tx = 0 ds_vlan_fwd_enq = 0 ds_drop_new_flow = 0 ds_flow_evict = 0 ds_trap_original = 0 ds_no_memory = 0 ds_fragment_queue_fail = 0 ds_no_frag_entry = 0 ds_icmp_error = 0 ds_clone_fail = 0 ds_drop_pkts = 0 ], ] ] ] 2025-09-01 Mon 02:15:25:763.111 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848854357504, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: VrouterStats 2025-09-01 Mon 02:15:25:763.137 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848854357504, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:25:763.210 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848854357504, Pid 111888]: SANDESH: Send FAILED: 1756692925762363 [SYS_NOTICE]: VrouterControlStatsTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3992-1 raw_xmpp_stats= [ [ _iter306->first = 10.20.0.17 [ reconnects = 0 in_msgs = 0 out_msgs = 0 ], ] ] raw_ifmap_stats= [ [ _iter307->first = link_delete_parse_errors _iter307->second = 0, _iter307->first = link_update_parse_errors _iter307->second = 0, _iter307->first = node_delete_parse_errors _iter307->second = 0, _iter307->first = node_update_parse_errors _iter307->second = 0, ] ] raw_rt_table_size= [ [ _iter308->first = default-domain:default-project:ip-fabric:__default__ [ inet4_unicast = 4 inet4_multicast = 0 evpn = 0 bridge = 2 inet6_unicast = 2 ], _iter308->first = default-domain:default-project:ip-fabric:ip-fabric [ inet4_unicast = 2 inet4_multicast = 0 evpn = 1 bridge = 2 inet6_unicast = 0 ], ] ] ] 2025-09-01 Mon 02:15:25:763.239 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848854357504, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: VrouterControlStatsTrace 2025-09-01 Mon 02:15:26:822.036 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848866952704, Pid 111888]: Processing scm::EvIdleHoldTimerExpired in state Idle 2025-09-01 Mon 02:15:26:822.454 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848866952704, Pid 111888]: Connect : Start Connect timer 10.0.0.254:8086 2025-09-01 Mon 02:15:26:822.529 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848886536832, Pid 111888]: -(-1) OnSessionEvent TCP Connect Failed 2025-09-01 Mon 02:15:26:822.646 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848886536832, Pid 111888]: -(-1):EvTcpConnectFail 2025-09-01 Mon 02:15:26:822.662 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848866952704, Pid 111888]: Processing scm::EvSandeshSend in state Connect 2025-09-01 Mon 02:15:26:822.751 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848866952704, Pid 111888]: SANDESH: Send FAILED: 1756692926822189 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-3992-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.17 server_addrs= [ [ (*_iter6) = 10.20.0.17:5269, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.17 server_addrs= [ [ (*_iter6) = 10.20.0.17:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.254:8086, ] ] status = Initializing description = Idle : EvIdleHoldTimerExpired -> Connect ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.17, XMPP:dns-server:10.20.0.17, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:15:26:822.769 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848866952704, Pid 111888]: Wrong state: Connect for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:15:26:822.789 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848866952704, Pid 111888]: Processing scm::EvSandeshSend in state Connect 2025-09-01 Mon 02:15:26:823.074 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848866952704, Pid 111888]: SANDESH: Send FAILED: 1756692926822634 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3992-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Connect successful_connections = 0 pid = 111888 http_port = 8085 start_time = 1756692145743123 collector_name = collector_ip = 10.0.0.254:8086 collector_list= [ [ (*_iter40) = 10.0.0.254:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ComputeCpuStateTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 13 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 65 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 114 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 176 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterControlStatsTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 26 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 39 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 244 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 62 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 241, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 506, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:15:26:823.122 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848866952704, Pid 111888]: Wrong state: Connect for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:15:26:823.144 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848866952704, Pid 111888]: Processing scm::EvTcpConnectFail in state Connect 2025-09-01 Mon 02:15:26:823.159 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848866952704, Pid 111888]: Connect : EvTcpConnectFail 2025-09-01 Mon 02:15:26:823.242 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848866952704, Pid 111888]: TCP [SYS_ERR]: TcpSessionMessageLog: Session -(-1) > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:15:26:823.297 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848866952704, Pid 111888]: Idle 2025-09-01 Mon 02:15:26:823.398 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848866952704, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:26:823.437 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848866952704, Pid 111888]: SANDESH: Send FAILED: 1756692926823200 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-3992-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.17 server_addrs= [ [ (*_iter6) = 10.20.0.17:5269, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.17 server_addrs= [ [ (*_iter6) = 10.20.0.17:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.17, XMPP:dns-server:10.20.0.17, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:15:26:823.452 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848866952704, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:15:26:823.470 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848866952704, Pid 111888]: Processing scm::EvTcpDeleteSession in state Idle 2025-09-01 Mon 02:15:26:823.515 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848866952704, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:26:823.802 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848866952704, Pid 111888]: SANDESH: Send FAILED: 1756692926823379 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3992-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Idle successful_connections = 0 pid = 111888 http_port = 8085 start_time = 1756692145743123 collector_name = collector_ip = 10.0.0.254:8086 collector_list= [ [ (*_iter40) = 10.0.0.254:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ComputeCpuStateTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 13 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 66 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 115 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 177 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterControlStatsTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 26 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 39 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 244 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 62 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 242, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 508, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:15:26:823.841 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848866952704, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:15:36:486.093 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848886536832, Pid 111888]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.20.0.17:8085::10.20.0.17:43462 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 255 2025-09-01 Mon 02:15:36:498.829 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848886536832, Pid 111888]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.20.0.17:8085::10.20.0.17:43474 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 255 2025-09-01 Mon 02:15:37:029.247 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848886536832, Pid 111888]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.0.0.254:8085::10.0.0.254:38756 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 255 2025-09-01 Mon 02:15:39:753.827 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848886536832, Pid 111888]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.0.0.254:8085::10.0.0.254:38760 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 255 2025-09-01 Mon 02:15:40:635.221 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:40:635.347 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: SANDESH: Send FAILED: 1756692940635163 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.17 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:40:635.490 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:40:635.527 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:40:635.572 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: SANDESH: Send FAILED: 1756692940635394 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.17 state = Connect event = Connect Event local_port = 38871 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:40:635.588 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:40:635.496 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848886536832, Pid 111888]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.17 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:15:40:635.791 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848854357504, Pid 111888]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:15:40:635.876 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:40:635.906 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: SANDESH: Send FAILED: 1756692940635859 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.17 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:40:635.919 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:40:635.935 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:40:635.955 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: SANDESH: Send FAILED: 1756692940635869 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.17 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:40:635.968 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:42:652.084 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848886536832, Pid 111888]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.0.0.254:8085::10.0.0.254:38770 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 255 2025-09-01 Mon 02:15:52:321.876 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848862754304, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:52:322.042 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848862754304, Pid 111888]: SANDESH: Send FAILED: 1756692952321794 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.17 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:52:322.121 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848862754304, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:52:322.450 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848886536832, Pid 111888]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.17 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:15:52:322.460 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848866952704, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:52:322.621 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848862754304, Pid 111888]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:15:52:322.663 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848866952704, Pid 111888]: SANDESH: Send FAILED: 1756692952322376 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.17 state = Connect event = Connect Event local_port = 40957 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:52:322.691 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848866952704, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:52:322.718 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848866952704, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:52:322.748 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848866952704, Pid 111888]: SANDESH: Send FAILED: 1756692952322696 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.17 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:52:322.766 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848866952704, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:52:322.791 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848866952704, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:52:322.833 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848866952704, Pid 111888]: SANDESH: Send FAILED: 1756692952322711 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.17 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:15:52:322.861 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848866952704, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:15:55:758.939 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:55:759.141 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: SANDESH: Send FAILED: 1756692955758609 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3992-1:Compute:contrail-vrouter-agent:0 sm_queue_count = 0 sm_stats= [ ev_stats= [ [ [ event = scm::EvCollectorUpdate enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvIdleHoldTimerExpired enqueues = 30 dequeues = 30 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvSandeshSend enqueues = 518 dequeues = 518 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvStart enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpConnectFail enqueues = 31 dequeues = 31 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpDeleteSession enqueues = 31 dequeues = 31 enqueue_fails = 0 dequeue_fails = 0 ], [ event = enqueues = 612 dequeues = 612 enqueue_fails = 0 dequeue_fails = 0 ], ] ] state = Idle state_since = 1756692926823295 last_state = Idle last_event = scm::EvSandeshSend last_event_at = 1756692952322785 ] max_sm_queue_count = 6 ] 2025-09-01 Mon 02:15:55:759.208 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:15:55:759.235 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:55:759.689 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: SANDESH: Send FAILED: 1756692955758821 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3992-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Idle successful_connections = 0 pid = 111888 http_port = 8085 start_time = 1756692145743123 collector_name = collector_ip = 10.0.0.254:8086 collector_list= [ [ (*_iter40) = 10.0.0.254:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ComputeCpuStateTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 13 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 67 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 116 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 184 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterControlStatsTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 26 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 39 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 252 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 64 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 251, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 518, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:15:55:759.743 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:15:55:762.794 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:55:763.177 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: SANDESH: Send FAILED: 1756692955762740 [SYS_NOTICE]: VrouterStats: data= [ name = cn-jenkins-deploy-platform-ansible-os-3992-1 in_tpkts = 208 in_bytes = 8932 out_tpkts = 228 out_bytes = 10488 raw_vhost_stats= [ name = vhost0 in_pkts = 208 in_bytes = 8932 out_pkts = 228 out_bytes = 10488 speed = -1 duplexity = -1 ] raw_vhost_drop_stats= [ ds_discard = 0 ds_pull = 0 ds_invalid_if = 0 ds_invalid_arp = 0 ds_trap_no_if = 0 ds_nowhere_to_go = 0 ds_flow_queue_limit_exceeded = 0 ds_flow_no_memory = 0 ds_flow_invalid_protocol = 0 ds_flow_nat_no_rflow = 0 ds_flow_action_drop = 0 ds_flow_action_invalid = 0 ds_flow_unusable = 0 ds_flow_table_full = 0 ds_interface_tx_discard = 0 ds_interface_drop = 0 ds_duplicated = 0 ds_push = 0 ds_ttl_exceeded = 0 ds_invalid_nh = 0 ds_invalid_label = 0 ds_invalid_protocol = 0 ds_interface_rx_discard = 0 ds_invalid_mcast_source = 0 ds_head_alloc_fail = 0 ds_pcow_fail = 0 ds_mcast_clone_fail = 0 ds_rewrite_fail = 0 ds_misc = 0 ds_invalid_packet = 0 ds_cksum_err = 0 ds_no_fmd = 0 ds_invalid_vnid = 0 ds_frag_err = 0 ds_invalid_source = 0 ds_mcast_df_bit = 0 ds_l2_no_route = 0 ds_vlan_fwd_tx = 0 ds_vlan_fwd_enq = 0 ds_drop_new_flow = 0 ds_flow_evict = 0 ds_trap_original = 0 ds_no_memory = 0 ds_fragment_queue_fail = 0 ds_no_frag_entry = 0 ds_icmp_error = 0 ds_clone_fail = 0 ds_drop_pkts = 0 ] raw_drop_stats= [ ds_discard = 0 ds_pull = 0 ds_invalid_if = 0 ds_invalid_arp = 0 ds_trap_no_if = 0 ds_nowhere_to_go = 0 ds_flow_queue_limit_exceeded = 0 ds_flow_no_memory = 0 ds_flow_invalid_protocol = 0 ds_flow_nat_no_rflow = 0 ds_flow_action_drop = 0 ds_flow_action_invalid = 0 ds_flow_unusable = 0 ds_flow_table_full = 0 ds_interface_tx_discard = 0 ds_interface_drop = 0 ds_duplicated = 0 ds_push = 0 ds_ttl_exceeded = 0 ds_invalid_nh = 0 ds_invalid_label = 0 ds_invalid_protocol = 0 ds_interface_rx_discard = 0 ds_invalid_mcast_source = 0 ds_head_alloc_fail = 0 ds_pcow_fail = 0 ds_mcast_clone_fail = 0 ds_rewrite_fail = 0 ds_misc = 0 ds_invalid_packet = 0 ds_cksum_err = 0 ds_no_fmd = 0 ds_invalid_vnid = 0 ds_frag_err = 0 ds_invalid_source = 0 ds_mcast_df_bit = 0 ds_l2_no_route = 0 ds_vlan_fwd_tx = 0 ds_vlan_fwd_enq = 0 ds_drop_new_flow = 0 ds_flow_evict = 0 ds_trap_original = 0 ds_no_memory = 0 ds_fragment_queue_fail = 0 ds_no_frag_entry = 0 ds_icmp_error = 0 ds_clone_fail = 0 ds_drop_pkts = 0 ] flow_rate= [ added_flows = 0 max_flow_adds_per_second = 0 min_flow_adds_per_second = 0 deleted_flows = 0 max_flow_deletes_per_second = 0 min_flow_deletes_per_second = 0 active_flows = 0 hold_flows = 0 ] phy_band_in_bps= [ [ _iter260->first = ens6 _iter260->second = 0, ] ] phy_band_out_bps= [ [ _iter261->first = ens6 _iter261->second = 33, ] ] raw_phy_if_stats= [ [ _iter262->first = ens6 [ in_pkts = 27 in_bytes = 2046 out_pkts = 208 out_bytes = 8932 ], ] ] raw_phy_if_drop_stats= [ [ _iter263->first = ens6 [ ds_discard = 0 ds_pull = 0 ds_invalid_if = 0 ds_invalid_arp = 0 ds_trap_no_if = 0 ds_nowhere_to_go = 0 ds_flow_queue_limit_exceeded = 0 ds_flow_no_memory = 0 ds_flow_invalid_protocol = 0 ds_flow_nat_no_rflow = 0 ds_flow_action_drop = 0 ds_flow_action_invalid = 0 ds_flow_unusable = 0 ds_flow_table_full = 0 ds_interface_tx_discard = 0 ds_interface_drop = 0 ds_duplicated = 0 ds_push = 0 ds_ttl_exceeded = 0 ds_invalid_nh = 0 ds_invalid_label = 0 ds_invalid_protocol = 0 ds_interface_rx_discard = 0 ds_invalid_mcast_source = 0 ds_head_alloc_fail = 0 ds_pcow_fail = 0 ds_mcast_clone_fail = 0 ds_rewrite_fail = 0 ds_misc = 0 ds_invalid_packet = 0 ds_cksum_err = 0 ds_no_fmd = 0 ds_invalid_vnid = 0 ds_frag_err = 0 ds_invalid_source = 0 ds_mcast_df_bit = 0 ds_l2_no_route = 0 ds_vlan_fwd_tx = 0 ds_vlan_fwd_enq = 0 ds_drop_new_flow = 0 ds_flow_evict = 0 ds_trap_original = 0 ds_no_memory = 0 ds_fragment_queue_fail = 0 ds_no_frag_entry = 0 ds_icmp_error = 0 ds_clone_fail = 0 ds_drop_pkts = 0 ], ] ] ] 2025-09-01 Mon 02:15:55:763.232 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: VrouterStats 2025-09-01 Mon 02:15:55:763.255 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:55:763.310 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: SANDESH: Send FAILED: 1756692955762850 [SYS_NOTICE]: VrouterControlStatsTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3992-1 raw_xmpp_stats= [ [ _iter306->first = 10.20.0.17 [ reconnects = 0 in_msgs = 0 out_msgs = 0 ], ] ] raw_ifmap_stats= [ [ _iter307->first = link_delete_parse_errors _iter307->second = 0, _iter307->first = link_update_parse_errors _iter307->second = 0, _iter307->first = node_delete_parse_errors _iter307->second = 0, _iter307->first = node_update_parse_errors _iter307->second = 0, ] ] raw_rt_table_size= [ [ _iter308->first = default-domain:default-project:ip-fabric:__default__ [ inet4_unicast = 4 inet4_multicast = 0 evpn = 0 bridge = 2 inet6_unicast = 2 ], _iter308->first = default-domain:default-project:ip-fabric:ip-fabric [ inet4_unicast = 2 inet4_multicast = 0 evpn = 1 bridge = 2 inet6_unicast = 0 ], ] ] ] 2025-09-01 Mon 02:15:55:763.326 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: VrouterControlStatsTrace 2025-09-01 Mon 02:15:56:823.494 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: Processing scm::EvIdleHoldTimerExpired in state Idle 2025-09-01 Mon 02:15:56:823.802 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: Connect : Start Connect timer 10.0.0.254:8086 2025-09-01 Mon 02:15:56:823.890 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848886536832, Pid 111888]: -(-1) OnSessionEvent TCP Connect Failed 2025-09-01 Mon 02:15:56:823.942 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: Processing scm::EvSandeshSend in state Connect 2025-09-01 Mon 02:15:56:823.981 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848886536832, Pid 111888]: -(-1):EvTcpConnectFail 2025-09-01 Mon 02:15:56:824.023 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: SANDESH: Send FAILED: 1756692956823614 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-3992-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.17 server_addrs= [ [ (*_iter6) = 10.20.0.17:5269, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.17 server_addrs= [ [ (*_iter6) = 10.20.0.17:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.254:8086, ] ] status = Initializing description = Idle : EvIdleHoldTimerExpired -> Connect ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.17, XMPP:dns-server:10.20.0.17, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:15:56:824.097 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: Wrong state: Connect for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:15:56:824.113 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: Processing scm::EvSandeshSend in state Connect 2025-09-01 Mon 02:15:56:824.280 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: SANDESH: Send FAILED: 1756692956823908 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3992-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Connect successful_connections = 0 pid = 111888 http_port = 8085 start_time = 1756692145743123 collector_name = collector_ip = 10.0.0.254:8086 collector_list= [ [ (*_iter40) = 10.0.0.254:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ComputeCpuStateTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 13 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 67 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 118 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 184 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterControlStatsTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 27 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 40 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 252 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 64 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 251, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 522, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:15:56:824.302 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: Wrong state: Connect for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:15:56:824.313 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: Processing scm::EvTcpConnectFail in state Connect 2025-09-01 Mon 02:15:56:824.321 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: Connect : EvTcpConnectFail 2025-09-01 Mon 02:15:56:824.371 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: TCP [SYS_ERR]: TcpSessionMessageLog: Session -(-1) > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:15:56:824.405 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: Idle 2025-09-01 Mon 02:15:56:824.480 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:56:824.506 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: SANDESH: Send FAILED: 1756692956824346 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-3992-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.17 server_addrs= [ [ (*_iter6) = 10.20.0.17:5269, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.17 server_addrs= [ [ (*_iter6) = 10.20.0.17:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.17, XMPP:dns-server:10.20.0.17, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:15:56:824.518 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:15:56:824.537 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: Processing scm::EvTcpDeleteSession in state Idle 2025-09-01 Mon 02:15:56:824.610 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:15:56:824.759 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: SANDESH: Send FAILED: 1756692956824459 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3992-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Idle successful_connections = 0 pid = 111888 http_port = 8085 start_time = 1756692145743123 collector_name = collector_ip = 10.0.0.254:8086 collector_list= [ [ (*_iter40) = 10.0.0.254:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ComputeCpuStateTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 13 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 68 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 119 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 185 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterControlStatsTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 27 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 40 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 252 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 64 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 252, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 524, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:15:56:824.779 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848871151104, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:16:11:416.168 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848866952704, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:16:11:416.252 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848866952704, Pid 111888]: SANDESH: Send FAILED: 1756692971416132 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.17 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:16:11:416.264 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848866952704, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:16:11:416.591 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848854357504, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:16:11:416.592 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848886536832, Pid 111888]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.17 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:16:11:416.702 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848854357504, Pid 111888]: SANDESH: Send FAILED: 1756692971416497 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.17 state = Connect event = Connect Event local_port = 35185 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:16:11:416.832 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848854357504, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:16:11:416.889 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:16:11:416.954 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848854357504, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:16:11:416.989 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848854357504, Pid 111888]: SANDESH: Send FAILED: 1756692971416940 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.17 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:16:11:417.008 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848854357504, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:16:11:417.050 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848854357504, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:16:11:417.107 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848854357504, Pid 111888]: SANDESH: Send FAILED: 1756692971416949 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.17 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:16:11:417.126 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848854357504, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:16:17:039.378 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848886536832, Pid 111888]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.0.0.254:8085::10.0.0.254:53844 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 255 2025-09-01 Mon 02:16:20:943.041 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848854357504, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:16:20:943.142 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848854357504, Pid 111888]: SANDESH: Send FAILED: 1756692980942991 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.17 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:16:20:943.167 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848854357504, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:16:20:943.357 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848866952704, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:16:20:943.399 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848886536832, Pid 111888]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.20.0.17 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-09-01 Mon 02:16:20:943.467 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848866952704, Pid 111888]: SANDESH: Send FAILED: 1756692980943290 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.17 state = Connect event = Connect Event local_port = 45821 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:16:20:943.621 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848866952704, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:16:20:943.657 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848862754304, Pid 111888]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:16:20:943.862 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848879547904, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:16:20:943.964 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848879547904, Pid 111888]: SANDESH: Send FAILED: 1756692980943718 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.17 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:16:20:944.002 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848879547904, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:16:20:944.030 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848879547904, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:16:20:944.060 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848879547904, Pid 111888]: SANDESH: Send FAILED: 1756692980943762 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.17 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-09-01 Mon 02:16:20:944.102 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848879547904, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-09-01 Mon 02:16:25:759.499 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848866952704, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:16:25:759.702 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848866952704, Pid 111888]: SANDESH: Send FAILED: 1756692985759084 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3992-1:Compute:contrail-vrouter-agent:0 sm_queue_count = 0 sm_stats= [ ev_stats= [ [ [ event = scm::EvCollectorUpdate enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvIdleHoldTimerExpired enqueues = 31 dequeues = 31 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvSandeshSend enqueues = 534 dequeues = 534 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvStart enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpConnectFail enqueues = 32 dequeues = 32 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpDeleteSession enqueues = 32 dequeues = 32 enqueue_fails = 0 dequeue_fails = 0 ], [ event = enqueues = 631 dequeues = 631 enqueue_fails = 0 dequeue_fails = 0 ], ] ] state = Idle state_since = 1756692956824404 last_state = Idle last_event = scm::EvSandeshSend last_event_at = 1756692980944026 ] max_sm_queue_count = 6 ] 2025-09-01 Mon 02:16:25:759.741 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848866952704, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:16:25:759.771 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848866952704, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:16:25:760.275 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848866952704, Pid 111888]: SANDESH: Send FAILED: 1756692985759350 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3992-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Idle successful_connections = 0 pid = 111888 http_port = 8085 start_time = 1756692145743123 collector_name = collector_ip = 10.0.0.254:8086 collector_list= [ [ (*_iter40) = 10.0.0.254:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ComputeCpuStateTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 13 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 69 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 120 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 188 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterControlStatsTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 27 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 40 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 260 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 66 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 257, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 534, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:16:25:760.383 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848866952704, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:16:25:763.340 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848879547904, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:16:25:763.412 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848879547904, Pid 111888]: SANDESH: Send FAILED: 1756692985763267 [SYS_NOTICE]: VrouterStats: data= [ name = cn-jenkins-deploy-platform-ansible-os-3992-1 cpu_info= [ num_cpu = 8 sys_mem_info= [ total = 32 used = 0 free = 0 buffers = 0 cached = 510398136 node_type = ] meminfo= [ virt = 32561 peakvirt = 0 res = 0 ] cpuload= [ one_min_avg = 0.19875 five_min_avg = 0.19 fifteen_min_avg = 0.2075 ] cpu_share = 0.316667 ] ] 2025-09-01 Mon 02:16:25:763.426 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848879547904, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: VrouterStats 2025-09-01 Mon 02:16:25:763.438 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848879547904, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:16:25:763.460 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848879547904, Pid 111888]: SANDESH: Send FAILED: 1756692985763328 [SYS_NOTICE]: ComputeCpuStateTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3992-1 cpu_info= [ [ [ mem_virt = 32561 cpu_share = 0.316667 used_sys_mem = 3784569160 one_min_cpuload = 0.19875 mem_res = 0 ], ] ] ] 2025-09-01 Mon 02:16:25:763.481 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848879547904, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: ComputeCpuStateTrace 2025-09-01 Mon 02:16:25:763.491 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848879547904, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:16:25:763.670 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848879547904, Pid 111888]: SANDESH: Send FAILED: 1756692985763376 [SYS_NOTICE]: VrouterStats: data= [ name = cn-jenkins-deploy-platform-ansible-os-3992-1 in_tpkts = 211 in_bytes = 9058 out_tpkts = 231 out_bytes = 10614 raw_vhost_stats= [ name = vhost0 in_pkts = 211 in_bytes = 9058 out_pkts = 231 out_bytes = 10614 speed = -1 duplexity = -1 ] raw_vhost_drop_stats= [ ds_discard = 0 ds_pull = 0 ds_invalid_if = 0 ds_invalid_arp = 0 ds_trap_no_if = 0 ds_nowhere_to_go = 0 ds_flow_queue_limit_exceeded = 0 ds_flow_no_memory = 0 ds_flow_invalid_protocol = 0 ds_flow_nat_no_rflow = 0 ds_flow_action_drop = 0 ds_flow_action_invalid = 0 ds_flow_unusable = 0 ds_flow_table_full = 0 ds_interface_tx_discard = 0 ds_interface_drop = 0 ds_duplicated = 0 ds_push = 0 ds_ttl_exceeded = 0 ds_invalid_nh = 0 ds_invalid_label = 0 ds_invalid_protocol = 0 ds_interface_rx_discard = 0 ds_invalid_mcast_source = 0 ds_head_alloc_fail = 0 ds_pcow_fail = 0 ds_mcast_clone_fail = 0 ds_rewrite_fail = 0 ds_misc = 0 ds_invalid_packet = 0 ds_cksum_err = 0 ds_no_fmd = 0 ds_invalid_vnid = 0 ds_frag_err = 0 ds_invalid_source = 0 ds_mcast_df_bit = 0 ds_l2_no_route = 0 ds_vlan_fwd_tx = 0 ds_vlan_fwd_enq = 0 ds_drop_new_flow = 0 ds_flow_evict = 0 ds_trap_original = 0 ds_no_memory = 0 ds_fragment_queue_fail = 0 ds_no_frag_entry = 0 ds_icmp_error = 0 ds_clone_fail = 0 ds_drop_pkts = 0 ] raw_drop_stats= [ ds_discard = 0 ds_pull = 0 ds_invalid_if = 0 ds_invalid_arp = 0 ds_trap_no_if = 0 ds_nowhere_to_go = 0 ds_flow_queue_limit_exceeded = 0 ds_flow_no_memory = 0 ds_flow_invalid_protocol = 0 ds_flow_nat_no_rflow = 0 ds_flow_action_drop = 0 ds_flow_action_invalid = 0 ds_flow_unusable = 0 ds_flow_table_full = 0 ds_interface_tx_discard = 0 ds_interface_drop = 0 ds_duplicated = 0 ds_push = 0 ds_ttl_exceeded = 0 ds_invalid_nh = 0 ds_invalid_label = 0 ds_invalid_protocol = 0 ds_interface_rx_discard = 0 ds_invalid_mcast_source = 0 ds_head_alloc_fail = 0 ds_pcow_fail = 0 ds_mcast_clone_fail = 0 ds_rewrite_fail = 0 ds_misc = 0 ds_invalid_packet = 0 ds_cksum_err = 0 ds_no_fmd = 0 ds_invalid_vnid = 0 ds_frag_err = 0 ds_invalid_source = 0 ds_mcast_df_bit = 0 ds_l2_no_route = 0 ds_vlan_fwd_tx = 0 ds_vlan_fwd_enq = 0 ds_drop_new_flow = 0 ds_flow_evict = 0 ds_trap_original = 0 ds_no_memory = 0 ds_fragment_queue_fail = 0 ds_no_frag_entry = 0 ds_icmp_error = 0 ds_clone_fail = 0 ds_drop_pkts = 0 ] flow_rate= [ added_flows = 0 max_flow_adds_per_second = 0 min_flow_adds_per_second = 0 deleted_flows = 0 max_flow_deletes_per_second = 0 min_flow_deletes_per_second = 0 active_flows = 0 hold_flows = 0 ] raw_phy_if_stats= [ [ _iter262->first = ens6 [ in_pkts = 27 in_bytes = 2046 out_pkts = 211 out_bytes = 9058 ], ] ] raw_phy_if_drop_stats= [ [ _iter263->first = ens6 [ ds_discard = 0 ds_pull = 0 ds_invalid_if = 0 ds_invalid_arp = 0 ds_trap_no_if = 0 ds_nowhere_to_go = 0 ds_flow_queue_limit_exceeded = 0 ds_flow_no_memory = 0 ds_flow_invalid_protocol = 0 ds_flow_nat_no_rflow = 0 ds_flow_action_drop = 0 ds_flow_action_invalid = 0 ds_flow_unusable = 0 ds_flow_table_full = 0 ds_interface_tx_discard = 0 ds_interface_drop = 0 ds_duplicated = 0 ds_push = 0 ds_ttl_exceeded = 0 ds_invalid_nh = 0 ds_invalid_label = 0 ds_invalid_protocol = 0 ds_interface_rx_discard = 0 ds_invalid_mcast_source = 0 ds_head_alloc_fail = 0 ds_pcow_fail = 0 ds_mcast_clone_fail = 0 ds_rewrite_fail = 0 ds_misc = 0 ds_invalid_packet = 0 ds_cksum_err = 0 ds_no_fmd = 0 ds_invalid_vnid = 0 ds_frag_err = 0 ds_invalid_source = 0 ds_mcast_df_bit = 0 ds_l2_no_route = 0 ds_vlan_fwd_tx = 0 ds_vlan_fwd_enq = 0 ds_drop_new_flow = 0 ds_flow_evict = 0 ds_trap_original = 0 ds_no_memory = 0 ds_fragment_queue_fail = 0 ds_no_frag_entry = 0 ds_icmp_error = 0 ds_clone_fail = 0 ds_drop_pkts = 0 ], ] ] ] 2025-09-01 Mon 02:16:25:763.689 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848879547904, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: VrouterStats 2025-09-01 Mon 02:16:25:763.700 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848879547904, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:16:25:763.736 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848879547904, Pid 111888]: SANDESH: Send FAILED: 1756692985763424 [SYS_NOTICE]: VrouterControlStatsTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3992-1 raw_xmpp_stats= [ [ _iter306->first = 10.20.0.17 [ reconnects = 0 in_msgs = 0 out_msgs = 0 ], ] ] raw_ifmap_stats= [ [ _iter307->first = link_delete_parse_errors _iter307->second = 0, _iter307->first = link_update_parse_errors _iter307->second = 0, _iter307->first = node_delete_parse_errors _iter307->second = 0, _iter307->first = node_update_parse_errors _iter307->second = 0, ] ] raw_rt_table_size= [ [ _iter308->first = default-domain:default-project:ip-fabric:__default__ [ inet4_unicast = 4 inet4_multicast = 0 evpn = 0 bridge = 2 inet6_unicast = 2 ], _iter308->first = default-domain:default-project:ip-fabric:ip-fabric [ inet4_unicast = 2 inet4_multicast = 0 evpn = 1 bridge = 2 inet6_unicast = 0 ], ] ] ] 2025-09-01 Mon 02:16:25:763.746 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848879547904, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: VrouterControlStatsTrace 2025-09-01 Mon 02:16:26:824.734 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Processing scm::EvIdleHoldTimerExpired in state Idle 2025-09-01 Mon 02:16:26:825.244 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Connect : Start Connect timer 10.0.0.254:8086 2025-09-01 Mon 02:16:26:825.339 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848886536832, Pid 111888]: -(-1) OnSessionEvent TCP Connect Failed 2025-09-01 Mon 02:16:26:825.446 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848886536832, Pid 111888]: -(-1):EvTcpConnectFail 2025-09-01 Mon 02:16:26:825.526 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Processing scm::EvSandeshSend in state Connect 2025-09-01 Mon 02:16:26:825.627 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: SANDESH: Send FAILED: 1756692986824924 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-3992-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.17 server_addrs= [ [ (*_iter6) = 10.20.0.17:5269, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.17 server_addrs= [ [ (*_iter6) = 10.20.0.17:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.254:8086, ] ] status = Initializing description = Idle : EvIdleHoldTimerExpired -> Connect ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.17, XMPP:dns-server:10.20.0.17, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:16:26:825.700 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Wrong state: Connect for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:16:26:825.731 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Processing scm::EvTcpConnectFail in state Connect 2025-09-01 Mon 02:16:26:825.773 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Connect : EvTcpConnectFail 2025-09-01 Mon 02:16:26:825.884 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: TCP [SYS_ERR]: TcpSessionMessageLog: Session -(-1) > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-09-01 Mon 02:16:26:825.957 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Idle 2025-09-01 Mon 02:16:26:826.101 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:16:26:826.598 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: SANDESH: Send FAILED: 1756692986825483 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3992-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Connect successful_connections = 0 pid = 111888 http_port = 8085 start_time = 1756692145743123 collector_name = collector_ip = 10.0.0.254:8086 collector_list= [ [ (*_iter40) = 10.0.0.254:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ComputeCpuStateTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 14 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 69 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 122 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 188 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterControlStatsTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 28 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 42 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 260 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 66 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 257, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 540, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:16:26:826.662 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-01 Mon 02:16:26:826.693 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:16:26:826.743 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: SANDESH: Send FAILED: 1756692986825829 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-3992-1 process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.20.0.17 server_addrs= [ [ (*_iter6) = 10.20.0.17:5269, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.20.0.17 server_addrs= [ [ (*_iter6) = 10.20.0.17:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.20.0.17, XMPP:dns-server:10.20.0.17, Collector connection down, No Configuration for self ], ] ] ] 2025-09-01 Mon 02:16:26:826.763 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-01 Mon 02:16:26:826.785 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Processing scm::EvTcpDeleteSession in state Idle 2025-09-01 Mon 02:16:26:826.869 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Processing scm::EvSandeshSend in state Idle 2025-09-01 Mon 02:16:26:827.235 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: SANDESH: Send FAILED: 1756692986826068 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-3992-1:Compute:contrail-vrouter-agent:0 client_info= [ status = Idle successful_connections = 0 pid = 111888 http_port = 8085 start_time = 1756692145743123 collector_name = collector_ip = 10.0.0.254:8086 collector_list= [ [ (*_iter40) = 10.0.0.254:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ComputeCpuStateTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 14 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 70 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 122 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 189 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterControlStatsTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 28 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 42 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 260 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 66 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 258, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 541, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-01 Mon 02:16:26:827.281 UTC cn-jenkins-deploy-platform-ansible-os-3992-1 [Thread 139848875349504, Pid 111888]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace