2025-12-11 Thu 01:43:20:897.514 UTC cn-jenkins-deploy-platform-k8s-manifests-278-1 [Thread 139889523475968, Pid 560]: KsyncTxQueue CPU pinning policy <>. KsyncTxQueuen not pinned to CPU 2025-12-11 Thu 01:43:21:006.921 UTC cn-jenkins-deploy-platform-k8s-manifests-278-1 [Thread 139889531872768, Pid 560]: SANDESH: No Client: 1765417401006863 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-k8s-manifests-278-1.cluster.local:Compute:contrail-vrouter-agent:0 client_info= [ status = Idle successful_connections = 0 pid = 560 http_port = 8085 start_time = 1765417401006781 collector_name = collector_ip = 0.0.0.0:0 collector_list= [ [ ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 0, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 1, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 0, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-12-11 Thu 01:43:21:007.901 UTC cn-jenkins-deploy-platform-k8s-manifests-278-1 [Thread 139889506682368, Pid 560]: SANDESH: Send FAILED: 1765417401007817 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-k8s-manifests-278-1.cluster.local process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Idle : EvStart -> Disconnect ], ] ] flag_infos= [ [ ] ] description = Collector connection down, No Configuration for self Number of connections:1, Expected: 3 ], ] ] ] 2025-12-11 Thu 01:43:21:008.122 UTC cn-jenkins-deploy-platform-k8s-manifests-278-1 [Thread 139889506682368, Pid 560]: SANDESH: Send FAILED: 1765417401007864 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-k8s-manifests-278-1.cluster.local:Compute:contrail-vrouter-agent:0 client_info= [ status = Disconnect successful_connections = 0 pid = 560 http_port = 8085 start_time = 1765417401006781 collector_name = collector_ip = 0.0.0.0:0 collector_list= [ [ ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 1, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 0, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-12-11 Thu 01:43:21:009.358 UTC cn-jenkins-deploy-platform-k8s-manifests-278-1 [Thread 139889510880768, Pid 560]: SANDESH: Send FAILED: 1765417401008941 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-k8s-manifests-278-1.cluster.local:Compute:contrail-vrouter-agent:0 client_info= [ status = Disconnect successful_connections = 0 pid = 560 http_port = 8085 start_time = 1765417401006781 collector_name = collector_ip = 10.0.0.22:8086 collector_list= [ [ (*_iter40) = 10.0.0.22:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 1, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 2, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-12-11 Thu 01:43:21:009.396 UTC cn-jenkins-deploy-platform-k8s-manifests-278-1 [Thread 139889510880768, Pid 560]: SANDESH: Send FAILED: 1765417401008963 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-k8s-manifests-278-1.cluster.local process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.22:8086, ] ] status = Initializing description = Disconnect : EvCollectorUpdate -> Connect ], ] ] flag_infos= [ [ ] ] description = Collector connection down, No Configuration for self Number of connections:1, Expected: 3 ], ] ] ] 2025-12-11 Thu 01:43:21:009.449 UTC cn-jenkins-deploy-platform-k8s-manifests-278-1 [Thread 139889510880768, Pid 560]: SANDESH: Send FAILED: 1765417401009269 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-k8s-manifests-278-1.cluster.local:Compute:contrail-vrouter-agent:0 client_info= [ status = Connect successful_connections = 0 pid = 560 http_port = 8085 start_time = 1765417401006781 collector_name = collector_ip = 10.0.0.22:8086 collector_list= [ [ (*_iter40) = 10.0.0.22:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 1, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 2, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-12-11 Thu 01:43:21:009.548 UTC cn-jenkins-deploy-platform-k8s-manifests-278-1 [Thread 139889519277568, Pid 560]: TCP [SYS_ERR]: TcpSessionMessageLog: Session -(-1) > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-12-11 Thu 01:43:21:009.620 UTC cn-jenkins-deploy-platform-k8s-manifests-278-1 [Thread 139889519277568, Pid 560]: SANDESH: Send FAILED: 1765417401009521 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-k8s-manifests-278-1.cluster.local process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = Collector connection down, No Configuration for self Number of connections:1, Expected: 3 ], ] ] ] 2025-12-11 Thu 01:43:21:009.708 UTC cn-jenkins-deploy-platform-k8s-manifests-278-1 [Thread 139889519277568, Pid 560]: SANDESH: Send FAILED: 1765417401009598 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-k8s-manifests-278-1.cluster.local:Compute:contrail-vrouter-agent:0 client_info= [ status = Idle successful_connections = 0 pid = 560 http_port = 8085 start_time = 1765417401006781 collector_name = collector_ip = 10.0.0.22:8086 collector_list= [ [ (*_iter40) = 10.0.0.22:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 3 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 2, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 5, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-12-11 Thu 01:43:21:009.946 UTC cn-jenkins-deploy-platform-k8s-manifests-278-1 [Thread 139889527674368, Pid 560]: SANDESH: Send FAILED: 1765417401009897 AgentVrf [SYS_INFO]: VrfObjectLog: vrf= [ event = Addition name = default-domain:default-project:ip-fabric:__default__ index = 0 ] file = controller/src/vnsw/agent/oper/vrf.cc line = 414 2025-12-11 Thu 01:43:21:009.961 UTC cn-jenkins-deploy-platform-k8s-manifests-278-1 [Thread 139889527674368, Pid 560]: SANDESH: Send FAILED: 1765417401009925 AgentVrf [SYS_INFO]: VrfObjectLog: vrf= [ event = Addition name = default-domain:default-project:ip-fabric:ip-fabric index = 1 ] file = controller/src/vnsw/agent/oper/vrf.cc line = 414 2025-12-11 Thu 01:43:21:027.899 UTC cn-jenkins-deploy-platform-k8s-manifests-278-1 [Thread 139889506682368, Pid 560]: SANDESH: Send FAILED: 1765417401027661 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.0.0.22 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-12-11 Thu 01:43:21:027.952 UTC cn-jenkins-deploy-platform-k8s-manifests-278-1 [Thread 139889506682368, Pid 560]: SANDESH: Send FAILED: 1765417401027719 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-k8s-manifests-278-1.cluster.local process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.0.0.22 server_addrs= [ [ (*_iter6) = 10.0.0.22:5269, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.0.0.22, Collector connection down, No Configuration for self Number of connections:2, Expected: 3 ], ] ] ] 2025-12-11 Thu 01:43:21:028.086 UTC cn-jenkins-deploy-platform-k8s-manifests-278-1 [Thread 139889515079168, Pid 560]: SANDESH: Send FAILED: 1765417401028056 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.0.0.22 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-12-11 Thu 01:43:21:028.114 UTC cn-jenkins-deploy-platform-k8s-manifests-278-1 [Thread 139889515079168, Pid 560]: SANDESH: Send FAILED: 1765417401028075 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-k8s-manifests-278-1.cluster.local process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.0.0.22 server_addrs= [ [ (*_iter6) = 10.0.0.22:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.0.0.22 server_addrs= [ [ (*_iter6) = 10.0.0.22:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.0.0.22, XMPP:dns-server:10.0.0.22, Collector connection down, No Configuration for self ], ] ] ] 2025-12-11 Thu 01:43:21:028.980 UTC cn-jenkins-deploy-platform-k8s-manifests-278-1 [Thread 139889515079168, Pid 560]: TCP [SYS_ERR]: TcpServerMessageLog: Server TCP bind(fe80::f816:3eff:fe5b:8eb3%vhost0:8097): Cannot assign requested address src/contrail-common/io/tcp_server.cc 110 2025-12-11 Thu 01:43:21:074.948 UTC cn-jenkins-deploy-platform-k8s-manifests-278-1 [Thread 139889515079168, Pid 560]: SANDESH: Send FAILED: 1765417401074655 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.0.0.22 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-12-11 Thu 01:43:21:074.989 UTC cn-jenkins-deploy-platform-k8s-manifests-278-1 [Thread 139889515079168, Pid 560]: SANDESH: Send FAILED: 1765417401074917 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.0.0.22 state = Connect event = Connect Event local_port = 45957 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-12-11 Thu 01:43:21:075.030 UTC cn-jenkins-deploy-platform-k8s-manifests-278-1 [Thread 139889543049216, Pid 560]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.0.0.22 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-12-11 Thu 01:43:21:075.075 UTC cn-jenkins-deploy-platform-k8s-manifests-278-1 [Thread 139889527674368, Pid 560]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-12-11 Thu 01:43:21:075.136 UTC cn-jenkins-deploy-platform-k8s-manifests-278-1 [Thread 139889527674368, Pid 560]: SANDESH: Send FAILED: 1765417401075104 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.0.0.22 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-12-11 Thu 01:43:21:080.507 UTC cn-jenkins-deploy-platform-k8s-manifests-278-1 [Thread 139889515079168, Pid 560]: SANDESH: Send FAILED: 1765417401080265 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.0.0.22 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-12-11 Thu 01:43:21:080.544 UTC cn-jenkins-deploy-platform-k8s-manifests-278-1 [Thread 139889515079168, Pid 560]: SANDESH: Send FAILED: 1765417401080481 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.0.0.22 state = Connect event = Connect Event local_port = 42981 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-12-11 Thu 01:43:21:080.826 UTC cn-jenkins-deploy-platform-k8s-manifests-278-1 [Thread 139889543049216, Pid 560]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.0.0.22 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-12-11 Thu 01:43:21:080.872 UTC cn-jenkins-deploy-platform-k8s-manifests-278-1 [Thread 139889536071168, Pid 560]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-12-11 Thu 01:43:21:080.949 UTC cn-jenkins-deploy-platform-k8s-manifests-278-1 [Thread 139889506682368, Pid 560]: SANDESH: Send FAILED: 1765417401080903 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.0.0.22 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-12-11 Thu 01:43:21:984.341 UTC cn-jenkins-deploy-platform-k8s-manifests-278-1 [Thread 139889536071168, Pid 560]: SANDESH: Send FAILED: 1765417401984228 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.0.0.22 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-12-11 Thu 01:43:21:984.975 UTC cn-jenkins-deploy-platform-k8s-manifests-278-1 [Thread 139889543049216, Pid 560]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.0.0.22 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-12-11 Thu 01:43:21:985.024 UTC cn-jenkins-deploy-platform-k8s-manifests-278-1 [Thread 139889527674368, Pid 560]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-12-11 Thu 01:43:21:985.120 UTC cn-jenkins-deploy-platform-k8s-manifests-278-1 [Thread 139889527674368, Pid 560]: SANDESH: Send FAILED: 1765417401985004 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.0.0.22 state = Connect event = Connect Event local_port = 41299 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-12-11 Thu 01:43:21:985.134 UTC cn-jenkins-deploy-platform-k8s-manifests-278-1 [Thread 139889527674368, Pid 560]: SANDESH: Send FAILED: 1765417401985051 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.0.0.22 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-12-11 Thu 01:43:22:010.217 UTC cn-jenkins-deploy-platform-k8s-manifests-278-1 [Thread 139889527674368, Pid 560]: SANDESH: Send FAILED: 1765417402009684 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-k8s-manifests-278-1.cluster.local process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.0.0.22 server_addrs= [ [ (*_iter6) = 10.0.0.22:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.0.0.22 server_addrs= [ [ (*_iter6) = 10.0.0.22:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.22:8086, ] ] status = Initializing description = Idle : EvIdleHoldTimerExpired -> Connect ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.0.0.22, XMPP:dns-server:10.0.0.22, Collector connection down, No Configuration for self ], ] ] ] 2025-12-11 Thu 01:43:22:010.296 UTC cn-jenkins-deploy-platform-k8s-manifests-278-1 [Thread 139889527674368, Pid 560]: TCP [SYS_ERR]: TcpSessionMessageLog: Session -(-1) > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-12-11 Thu 01:43:22:010.489 UTC cn-jenkins-deploy-platform-k8s-manifests-278-1 [Thread 139889527674368, Pid 560]: SANDESH: Send FAILED: 1765417402010183 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-k8s-manifests-278-1.cluster.local:Compute:contrail-vrouter-agent:0 client_info= [ status = Connect successful_connections = 0 pid = 560 http_port = 8085 start_time = 1765417401006781 collector_name = collector_ip = 10.0.0.22:8086 collector_list= [ [ (*_iter40) = 10.0.0.22:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 5 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 4 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 4 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 11 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 10, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 22, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-12-11 Thu 01:43:22:010.522 UTC cn-jenkins-deploy-platform-k8s-manifests-278-1 [Thread 139889527674368, Pid 560]: SANDESH: Send FAILED: 1765417402010276 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-k8s-manifests-278-1.cluster.local process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.0.0.22 server_addrs= [ [ (*_iter6) = 10.0.0.22:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.0.0.22 server_addrs= [ [ (*_iter6) = 10.0.0.22:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.0.0.22, XMPP:dns-server:10.0.0.22, Collector connection down, No Configuration for self ], ] ] ] 2025-12-11 Thu 01:43:22:010.631 UTC cn-jenkins-deploy-platform-k8s-manifests-278-1 [Thread 139889527674368, Pid 560]: SANDESH: Send FAILED: 1765417402010349 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-k8s-manifests-278-1.cluster.local:Compute:contrail-vrouter-agent:0 client_info= [ status = Idle successful_connections = 0 pid = 560 http_port = 8085 start_time = 1765417401006781 collector_name = collector_ip = 10.0.0.22:8086 collector_list= [ [ (*_iter40) = 10.0.0.22:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 6 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 4 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 5 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 11 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 11, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 23, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-12-11 Thu 01:43:22:025.998 UTC cn-jenkins-deploy-platform-k8s-manifests-278-1 [Thread 139889519277568, Pid 560]: SANDESH: Send FAILED: 1765417402025969 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.0.0.22 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-12-11 Thu 01:43:22:026.223 UTC cn-jenkins-deploy-platform-k8s-manifests-278-1 [Thread 139889543049216, Pid 560]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.0.0.22 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-12-11 Thu 01:43:22:026.236 UTC cn-jenkins-deploy-platform-k8s-manifests-278-1 [Thread 139889515079168, Pid 560]: SANDESH: Send FAILED: 1765417402026209 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.0.0.22 state = Connect event = Connect Event local_port = 39521 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-12-11 Thu 01:43:22:026.274 UTC cn-jenkins-deploy-platform-k8s-manifests-278-1 [Thread 139889536071168, Pid 560]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-12-11 Thu 01:43:22:026.306 UTC cn-jenkins-deploy-platform-k8s-manifests-278-1 [Thread 139889515079168, Pid 560]: SANDESH: Send FAILED: 1765417402026297 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.0.0.22 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-12-11 Thu 01:43:23:983.269 UTC cn-jenkins-deploy-platform-k8s-manifests-278-1 [Thread 139889531872768, Pid 560]: SANDESH: Send FAILED: 1765417403983214 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.0.0.22 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-12-11 Thu 01:43:23:983.981 UTC cn-jenkins-deploy-platform-k8s-manifests-278-1 [Thread 139889543049216, Pid 560]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.0.0.22 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-12-11 Thu 01:43:23:984.090 UTC cn-jenkins-deploy-platform-k8s-manifests-278-1 [Thread 139889515079168, Pid 560]: SANDESH: Send FAILED: 1765417403984028 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.0.0.22 state = Connect event = Connect Event local_port = 43771 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-12-11 Thu 01:43:23:984.269 UTC cn-jenkins-deploy-platform-k8s-manifests-278-1 [Thread 139889519277568, Pid 560]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-12-11 Thu 01:43:23:984.350 UTC cn-jenkins-deploy-platform-k8s-manifests-278-1 [Thread 139889510880768, Pid 560]: SANDESH: Send FAILED: 1765417403984304 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.0.0.22 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-12-11 Thu 01:43:24:011.309 UTC cn-jenkins-deploy-platform-k8s-manifests-278-1 [Thread 139889531872768, Pid 560]: SANDESH: Send FAILED: 1765417404010419 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-k8s-manifests-278-1.cluster.local process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.0.0.22 server_addrs= [ [ (*_iter6) = 10.0.0.22:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.0.0.22 server_addrs= [ [ (*_iter6) = 10.0.0.22:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.22:8086, ] ] status = Initializing description = Idle : EvIdleHoldTimerExpired -> Connect ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.0.0.22, XMPP:dns-server:10.0.0.22, Collector connection down, No Configuration for self ], ] ] ] 2025-12-11 Thu 01:43:24:011.471 UTC cn-jenkins-deploy-platform-k8s-manifests-278-1 [Thread 139889531872768, Pid 560]: TCP [SYS_ERR]: TcpSessionMessageLog: Session -(-1) > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-12-11 Thu 01:43:24:011.665 UTC cn-jenkins-deploy-platform-k8s-manifests-278-1 [Thread 139889531872768, Pid 560]: SANDESH: Send FAILED: 1765417404011269 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-k8s-manifests-278-1.cluster.local:Compute:contrail-vrouter-agent:0 client_info= [ status = Connect successful_connections = 0 pid = 560 http_port = 8085 start_time = 1765417401006781 collector_name = collector_ip = 10.0.0.22:8086 collector_list= [ [ (*_iter40) = 10.0.0.22:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 7 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 6 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 9 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 17 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 5 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 17, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 32, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-12-11 Thu 01:43:24:011.694 UTC cn-jenkins-deploy-platform-k8s-manifests-278-1 [Thread 139889531872768, Pid 560]: SANDESH: Send FAILED: 1765417404011445 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-k8s-manifests-278-1.cluster.local process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.0.0.22 server_addrs= [ [ (*_iter6) = 10.0.0.22:5269, ] ] status = Down description = Idle ], [ type = XMPP name = dns-server:10.0.0.22 server_addrs= [ [ (*_iter6) = 10.0.0.22:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.0.0.22, XMPP:dns-server:10.0.0.22, Collector connection down, No Configuration for self ], ] ] ] 2025-12-11 Thu 01:43:24:011.806 UTC cn-jenkins-deploy-platform-k8s-manifests-278-1 [Thread 139889531872768, Pid 560]: SANDESH: Send FAILED: 1765417404011535 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-k8s-manifests-278-1.cluster.local:Compute:contrail-vrouter-agent:0 client_info= [ status = Idle successful_connections = 0 pid = 560 http_port = 8085 start_time = 1765417401006781 collector_name = collector_ip = 10.0.0.22:8086 collector_list= [ [ (*_iter40) = 10.0.0.22:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 8 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 6 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 10 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 17 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 5 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 18, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 33, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-12-11 Thu 01:43:24:150.450 UTC cn-jenkins-deploy-platform-k8s-manifests-278-1 [Thread 139889531872768, Pid 560]: SANDESH: Send FAILED: 1765417404150405 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.0.0.22 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-12-11 Thu 01:43:24:150.889 UTC cn-jenkins-deploy-platform-k8s-manifests-278-1 [Thread 139889543049216, Pid 560]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.0.0.22 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-12-11 Thu 01:43:24:150.902 UTC cn-jenkins-deploy-platform-k8s-manifests-278-1 [Thread 139889527674368, Pid 560]: SANDESH: Send FAILED: 1765417404150869 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.0.0.22 state = Connect event = Connect Event local_port = 38625 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-12-11 Thu 01:43:24:150.985 UTC cn-jenkins-deploy-platform-k8s-manifests-278-1 [Thread 139889506682368, Pid 560]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-12-11 Thu 01:43:24:151.038 UTC cn-jenkins-deploy-platform-k8s-manifests-278-1 [Thread 139889510880768, Pid 560]: SANDESH: Send FAILED: 1765417404151026 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.0.0.22 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-12-11 Thu 01:43:27:692.557 UTC cn-jenkins-deploy-platform-k8s-manifests-278-1 [Thread 139889531872768, Pid 560]: SANDESH: Send FAILED: 1765417407692410 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.0.0.22 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-12-11 Thu 01:43:27:693.074 UTC cn-jenkins-deploy-platform-k8s-manifests-278-1 [Thread 139889543049216, Pid 560]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.0.0.22 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-12-11 Thu 01:43:27:693.137 UTC cn-jenkins-deploy-platform-k8s-manifests-278-1 [Thread 139889527674368, Pid 560]: SANDESH: Send FAILED: 1765417407693097 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.0.0.22 state = Connect event = Connect Event local_port = 44321 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-12-11 Thu 01:43:27:693.518 UTC cn-jenkins-deploy-platform-k8s-manifests-278-1 [Thread 139889515079168, Pid 560]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-12-11 Thu 01:43:27:693.638 UTC cn-jenkins-deploy-platform-k8s-manifests-278-1 [Thread 139889506682368, Pid 560]: SANDESH: Send FAILED: 1765417407693544 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.0.0.22 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-12-11 Thu 01:43:27:693.651 UTC cn-jenkins-deploy-platform-k8s-manifests-278-1 [Thread 139889506682368, Pid 560]: SANDESH: Send FAILED: 1765417407693552 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.0.0.22 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-12-11 Thu 01:43:27:693.676 UTC cn-jenkins-deploy-platform-k8s-manifests-278-1 [Thread 139889506682368, Pid 560]: SANDESH: Send FAILED: 1765417407693615 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-k8s-manifests-278-1.cluster.local process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.0.0.22 server_addrs= [ [ (*_iter6) = 10.0.0.22:5269, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.0.0.22 server_addrs= [ [ (*_iter6) = 10.0.0.22:53, ] ] status = Down description = Idle ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.0.0.22, XMPP:dns-server:10.0.0.22, Collector connection down, No Configuration for self ], ] ] ] 2025-12-11 Thu 01:43:27:895.271 UTC cn-jenkins-deploy-platform-k8s-manifests-278-1 [Thread 139889536071168, Pid 560]: SANDESH: Send FAILED: 1765417407895120 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.0.0.22 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-12-11 Thu 01:43:27:896.117 UTC cn-jenkins-deploy-platform-k8s-manifests-278-1 [Thread 139889543049216, Pid 560]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.0.0.22 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-12-11 Thu 01:43:27:896.162 UTC cn-jenkins-deploy-platform-k8s-manifests-278-1 [Thread 139889519277568, Pid 560]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-12-11 Thu 01:43:27:896.296 UTC cn-jenkins-deploy-platform-k8s-manifests-278-1 [Thread 139889519277568, Pid 560]: SANDESH: Send FAILED: 1765417407896141 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.0.0.22 state = Connect event = Connect Event local_port = 46529 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-12-11 Thu 01:43:27:896.312 UTC cn-jenkins-deploy-platform-k8s-manifests-278-1 [Thread 139889519277568, Pid 560]: SANDESH: Send FAILED: 1765417407896208 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.0.0.22 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-12-11 Thu 01:43:27:896.322 UTC cn-jenkins-deploy-platform-k8s-manifests-278-1 [Thread 139889519277568, Pid 560]: SANDESH: Send FAILED: 1765417407896213 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.0.0.22 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-12-11 Thu 01:43:27:896.343 UTC cn-jenkins-deploy-platform-k8s-manifests-278-1 [Thread 139889519277568, Pid 560]: SANDESH: Send FAILED: 1765417407896279 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-k8s-manifests-278-1.cluster.local process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.0.0.22 server_addrs= [ [ (*_iter6) = 10.0.0.22:5269, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.0.0.22 server_addrs= [ [ (*_iter6) = 10.0.0.22:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.0.0.22, XMPP:dns-server:10.0.0.22, Collector connection down, No Configuration for self ], ] ] ] 2025-12-11 Thu 01:43:28:012.628 UTC cn-jenkins-deploy-platform-k8s-manifests-278-1 [Thread 139889531872768, Pid 560]: SANDESH: Send FAILED: 1765417408011614 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-k8s-manifests-278-1.cluster.local process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.0.0.22 server_addrs= [ [ (*_iter6) = 10.0.0.22:5269, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.0.0.22 server_addrs= [ [ (*_iter6) = 10.0.0.22:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.22:8086, ] ] status = Initializing description = Idle : EvIdleHoldTimerExpired -> Connect ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.0.0.22, XMPP:dns-server:10.0.0.22, Collector connection down, No Configuration for self ], ] ] ] 2025-12-11 Thu 01:43:28:012.693 UTC cn-jenkins-deploy-platform-k8s-manifests-278-1 [Thread 139889531872768, Pid 560]: TCP [SYS_ERR]: TcpSessionMessageLog: Session -(-1) > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-12-11 Thu 01:43:28:012.894 UTC cn-jenkins-deploy-platform-k8s-manifests-278-1 [Thread 139889531872768, Pid 560]: SANDESH: Send FAILED: 1765417408012601 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-k8s-manifests-278-1.cluster.local:Compute:contrail-vrouter-agent:0 client_info= [ status = Connect successful_connections = 0 pid = 560 http_port = 8085 start_time = 1765417401006781 collector_name = collector_ip = 10.0.0.22:8086 collector_list= [ [ (*_iter40) = 10.0.0.22:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 11 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 8 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 13 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 28 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 8 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 24, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 49, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-12-11 Thu 01:43:28:012.920 UTC cn-jenkins-deploy-platform-k8s-manifests-278-1 [Thread 139889531872768, Pid 560]: SANDESH: Send FAILED: 1765417408012676 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-k8s-manifests-278-1.cluster.local process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.0.0.22 server_addrs= [ [ (*_iter6) = 10.0.0.22:5269, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.0.0.22 server_addrs= [ [ (*_iter6) = 10.0.0.22:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.0.0.22, XMPP:dns-server:10.0.0.22, Collector connection down, No Configuration for self ], ] ] ] 2025-12-11 Thu 01:43:28:013.013 UTC cn-jenkins-deploy-platform-k8s-manifests-278-1 [Thread 139889531872768, Pid 560]: SANDESH: Send FAILED: 1765417408012754 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-k8s-manifests-278-1.cluster.local:Compute:contrail-vrouter-agent:0 client_info= [ status = Idle successful_connections = 0 pid = 560 http_port = 8085 start_time = 1765417401006781 collector_name = collector_ip = 10.0.0.22:8086 collector_list= [ [ (*_iter40) = 10.0.0.22:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 12 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 8 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 14 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 28 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 8 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 25, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 50, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-12-11 Thu 01:43:35:672.343 UTC cn-jenkins-deploy-platform-k8s-manifests-278-1 [Thread 139889527674368, Pid 560]: SANDESH: Send FAILED: 1765417415672294 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.0.0.22 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-12-11 Thu 01:43:35:672.705 UTC cn-jenkins-deploy-platform-k8s-manifests-278-1 [Thread 139889515079168, Pid 560]: SANDESH: Send FAILED: 1765417415672670 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.0.0.22 state = Connect event = Connect Event local_port = 38497 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-12-11 Thu 01:43:35:672.705 UTC cn-jenkins-deploy-platform-k8s-manifests-278-1 [Thread 139889543049216, Pid 560]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.0.0.22 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-12-11 Thu 01:43:35:672.810 UTC cn-jenkins-deploy-platform-k8s-manifests-278-1 [Thread 139889506682368, Pid 560]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-12-11 Thu 01:43:35:672.869 UTC cn-jenkins-deploy-platform-k8s-manifests-278-1 [Thread 139889519277568, Pid 560]: SANDESH: Send FAILED: 1765417415672843 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.0.0.22 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-12-11 Thu 01:43:35:672.891 UTC cn-jenkins-deploy-platform-k8s-manifests-278-1 [Thread 139889519277568, Pid 560]: SANDESH: Send FAILED: 1765417415672851 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.0.0.22 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-12-11 Thu 01:43:35:757.661 UTC cn-jenkins-deploy-platform-k8s-manifests-278-1 [Thread 139889519277568, Pid 560]: SANDESH: Send FAILED: 1765417415757619 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.0.0.22 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-12-11 Thu 01:43:35:757.954 UTC cn-jenkins-deploy-platform-k8s-manifests-278-1 [Thread 139889510880768, Pid 560]: SANDESH: Send FAILED: 1765417415757920 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.0.0.22 state = Connect event = Connect Event local_port = 34255 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-12-11 Thu 01:43:35:757.955 UTC cn-jenkins-deploy-platform-k8s-manifests-278-1 [Thread 139889543049216, Pid 560]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.0.0.22 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-12-11 Thu 01:43:35:758.023 UTC cn-jenkins-deploy-platform-k8s-manifests-278-1 [Thread 139889506682368, Pid 560]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-12-11 Thu 01:43:35:758.488 UTC cn-jenkins-deploy-platform-k8s-manifests-278-1 [Thread 139889510880768, Pid 560]: SANDESH: Send FAILED: 1765417415758055 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.0.0.22 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-12-11 Thu 01:43:35:758.510 UTC cn-jenkins-deploy-platform-k8s-manifests-278-1 [Thread 139889510880768, Pid 560]: SANDESH: Send FAILED: 1765417415758060 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.0.0.22 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-12-11 Thu 01:43:36:015.347 UTC cn-jenkins-deploy-platform-k8s-manifests-278-1 [Thread 139889510880768, Pid 560]: SANDESH: Send FAILED: 1765417416014332 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-k8s-manifests-278-1.cluster.local process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.0.0.22 server_addrs= [ [ (*_iter6) = 10.0.0.22:5269, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.0.0.22 server_addrs= [ [ (*_iter6) = 10.0.0.22:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.22:8086, ] ] status = Initializing description = Idle : EvIdleHoldTimerExpired -> Connect ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.0.0.22, XMPP:dns-server:10.0.0.22, Collector connection down, No Configuration for self ], ] ] ] 2025-12-11 Thu 01:43:36:015.418 UTC cn-jenkins-deploy-platform-k8s-manifests-278-1 [Thread 139889510880768, Pid 560]: TCP [SYS_ERR]: TcpSessionMessageLog: Session -(-1) > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-12-11 Thu 01:43:36:015.589 UTC cn-jenkins-deploy-platform-k8s-manifests-278-1 [Thread 139889510880768, Pid 560]: SANDESH: Send FAILED: 1765417416015317 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-k8s-manifests-278-1.cluster.local:Compute:contrail-vrouter-agent:0 client_info= [ status = Connect successful_connections = 0 pid = 560 http_port = 8085 start_time = 1765417401006781 collector_name = collector_ip = 10.0.0.22:8086 collector_list= [ [ (*_iter40) = 10.0.0.22:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 13 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 10 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 20 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 36 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 10 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 33, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 61, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-12-11 Thu 01:43:36:015.624 UTC cn-jenkins-deploy-platform-k8s-manifests-278-1 [Thread 139889510880768, Pid 560]: SANDESH: Send FAILED: 1765417416015400 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-k8s-manifests-278-1.cluster.local process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.0.0.22 server_addrs= [ [ (*_iter6) = 10.0.0.22:5269, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.0.0.22 server_addrs= [ [ (*_iter6) = 10.0.0.22:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.0.0.22, XMPP:dns-server:10.0.0.22, Collector connection down, No Configuration for self ], ] ] ] 2025-12-11 Thu 01:43:36:015.721 UTC cn-jenkins-deploy-platform-k8s-manifests-278-1 [Thread 139889510880768, Pid 560]: SANDESH: Send FAILED: 1765417416015473 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-k8s-manifests-278-1.cluster.local:Compute:contrail-vrouter-agent:0 client_info= [ status = Idle successful_connections = 0 pid = 560 http_port = 8085 start_time = 1765417401006781 collector_name = collector_ip = 10.0.0.22:8086 collector_list= [ [ (*_iter40) = 10.0.0.22:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 14 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 10 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 21 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 36 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 10 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 34, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 62, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-12-11 Thu 01:43:50:073.518 UTC cn-jenkins-deploy-platform-k8s-manifests-278-1 [Thread 139889543049216, Pid 560]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.0.0.22 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-12-11 Thu 01:43:50:073.547 UTC cn-jenkins-deploy-platform-k8s-manifests-278-1 [Thread 139889506682368, Pid 560]: SANDESH: Send FAILED: 1765417430073084 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.0.0.22 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-12-11 Thu 01:43:50:073.651 UTC cn-jenkins-deploy-platform-k8s-manifests-278-1 [Thread 139889515079168, Pid 560]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-12-11 Thu 01:43:50:073.677 UTC cn-jenkins-deploy-platform-k8s-manifests-278-1 [Thread 139889506682368, Pid 560]: SANDESH: Send FAILED: 1765417430073472 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.0.0.22 state = Connect event = Connect Event local_port = 45835 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-12-11 Thu 01:43:50:073.766 UTC cn-jenkins-deploy-platform-k8s-manifests-278-1 [Thread 139889506682368, Pid 560]: SANDESH: Send FAILED: 1765417430073720 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.0.0.22 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-12-11 Thu 01:43:50:073.797 UTC cn-jenkins-deploy-platform-k8s-manifests-278-1 [Thread 139889506682368, Pid 560]: SANDESH: Send FAILED: 1765417430073763 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.0.0.22 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-12-11 Thu 01:43:50:761.523 UTC cn-jenkins-deploy-platform-k8s-manifests-278-1 [Thread 139889527674368, Pid 560]: SANDESH: Send FAILED: 1765417430761448 [SYS_NOTICE]: UveVirtualNetworkAgentTrace: data= [ name = __UNKNOWN__ vrf_stats_list= [ [ [ name = __untitled__ diag_packet_count = 0 unknown_unicast_floods = 0 arp_packet_counts= [ from_vm_interface= [ stats= [ proxies = 0 stitches = 0 floods = 0 ] ] from_physical_interface= [ stats= [ proxies = 0 stitches = 0 floods = 0 ] ] ] offload_packet_counts= [ gro = 0 ] nh_packet_counts= [ discards = 0 resolves = 0 l3_receives = 0 l2_receives = 0 local_vm_l3_forwards = 0 local_vm_l2_forwards = 0 vrf_translates = 0 ecmp_forwards = 0 tunnel_nh_stats= [ udp_encaps = 0 mpls_over_udp_encaps = 0 mpls_over_gre_encaps = 0 vxlan_encaps = 0 ] comp_nh_stats= [ source_replication_forwards = 0 edge_replication_forwards = 0 local_vm_l3_forwards = 0 total_multicast_forwards = 0 ] ] ], ] ] ] 2025-12-11 Thu 01:43:50:761.630 UTC cn-jenkins-deploy-platform-k8s-manifests-278-1 [Thread 139889527674368, Pid 560]: SANDESH: Send FAILED: 1765417430761599 [SYS_NOTICE]: UveVrouterAgent: data= [ name = cn-jenkins-deploy-platform-k8s-manifests-278-1.cluster.local interface_list= [ [ ] ] error_intf_list= [ [ ] ] no_config_intf_list= [ [ (*_iter151) = 00000000-0000-0000-0000-000000000000, ] ] vmi_count= [ active = 1 ] down_interface_count = 1 ] 2025-12-11 Thu 01:43:51:007.948 UTC cn-jenkins-deploy-platform-k8s-manifests-278-1 [Thread 139889527674368, Pid 560]: SANDESH: Send FAILED: 1765417431007802 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-k8s-manifests-278-1.cluster.local:Compute:contrail-vrouter-agent:0 sm_queue_count = 0 sm_stats= [ ev_stats= [ [ [ event = scm::EvCollectorUpdate enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvIdleHoldTimerExpired enqueues = 4 dequeues = 4 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvSandeshSend enqueues = 71 dequeues = 71 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvStart enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpConnectFail enqueues = 5 dequeues = 5 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpDeleteSession enqueues = 5 dequeues = 5 enqueue_fails = 0 dequeue_fails = 0 ], [ event = enqueues = 87 dequeues = 87 enqueue_fails = 0 dequeue_fails = 0 ], ] ] state = Idle state_since = 1765417416015443 last_state = Idle last_event = scm::EvSandeshSend last_event_at = 1765417430761610 ] max_sm_queue_count = 4 ] 2025-12-11 Thu 01:43:51:008.085 UTC cn-jenkins-deploy-platform-k8s-manifests-278-1 [Thread 139889527674368, Pid 560]: SANDESH: Send FAILED: 1765417431007894 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-k8s-manifests-278-1.cluster.local:Compute:contrail-vrouter-agent:0 client_info= [ status = Idle successful_connections = 0 pid = 560 http_port = 8085 start_time = 1765417401006781 collector_name = collector_ip = 10.0.0.22:8086 collector_list= [ [ (*_iter40) = 10.0.0.22:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 15 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 12 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 28 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 40 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 11 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 42, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 71, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-12-11 Thu 01:43:51:013.588 UTC cn-jenkins-deploy-platform-k8s-manifests-278-1 [Thread 139889531872768, Pid 560]: SANDESH: Send FAILED: 1765417431013428 [SYS_NOTICE]: UveVrouterAgent: data= [ name = cn-jenkins-deploy-platform-k8s-manifests-278-1.cluster.local xmpp_peer_list= [ [ [ ip = 10.0.0.22 status = 0 setup_time = 0 primary = 1 ], ] ] self_ip_list= [ [ (*_iter146) = 10.0.0.22, ] ] build_info = {"build-info": [{"build-version": "v0+nightly", "build-time": "2025-12-11 00:13:25.979200", "build-user": "root", "build-hostname": "jenkins-build-prepare-sandbox-13006-1.novalocal", "build-id": "v0+nightly", "build-number": "202512110009"}]} phy_if= [ [ [ name = eth0 mac_address = fa:16:3e:5b:8e:b3 ], ] ] vhost_if= [ name = vhost0 mac_address = fa:16:3e:5b:8e:b3 ] log_file = /var/log/contrail/vrouter-agent/contrail-vrouter-agent.log config_file = /etc/contrail/contrail-vrouter-agent.conf log_local = 1 log_level = SYS_NOTICE log_category = sandesh_http_port = 8085 dns_servers= [ [ (*_iter149) = 10.0.0.22, ] ] vhost_cfg= [ name = vhost0 ip = 10.0.0.22 ip_prefix_len = 24 gateway = 10.0.0.1 ] eth_name= [ [ (*_iter150) = eth0, ] ] tunnel_type = MPLSoGRE hypervisor = kvm control_ip = 10.0.0.22 dns_server_list_cfg= [ [ (*_iter152) = 10.0.0.22:53, ] ] control_node_list_cfg= [ [ (*_iter153) = 10.0.0.22:5269, ] ] hostname_cfg = cn-jenkins-deploy-platform-k8s-manifests-278-1.cluster.local flow_cache_timeout_cfg = 0 ll_max_system_flows_cfg = 16384 ll_max_vm_flows_cfg = 16384 gateway_cfg_list= [ [ ] ] headless_mode_cfg = 1 collector_server_list_cfg= [ [ (*_iter155) = 10.0.0.22:8086, ] ] log_flow = 0 mode = VROUTER platform = HOST vr_limits= [ max_labels = 5120 max_nexthops = 524288 max_interfaces = 4355 max_vrfs = 4096 max_mirror_entries = 255 vrouter_max_bridge_entries = 262144 vrouter_max_oflow_bridge_entries = 53248 vrouter_build_info = {"build-info": [{"build-version": "v0+nightly", "build-time": "2025-12-11 00:10:00.884943", "build-user": "root", "build-hostname": "jenkins-build-prepare-sandbox-13006-1.novalocal", "build-id": "v0+nightly", "build-number": "202512110009"}]} vrouter_max_flow_entries = 524288 vrouter_max_oflow_entries = 105472 vrouter_priority_tagging = 1 ] bgpaas_enabled = 0 port_mirror_enabled = 0 subcluster_name = vr_high_watermark = 80 vr_low_watermark = 75 vr_limit_exceeded_list= [ [ ] ] res_limit = 0 res_table_limit = 0 loopback_ip = 0.0.0.0 gateway_list= [ [ (*_iter161) = 10.0.0.1, ] ] ] 2025-12-11 Thu 01:43:51:013.721 UTC cn-jenkins-deploy-platform-k8s-manifests-278-1 [Thread 139889531872768, Pid 560]: SANDESH: Send FAILED: 1765417431013470 [SYS_NOTICE]: VrouterStats: data= [ name = cn-jenkins-deploy-platform-k8s-manifests-278-1.cluster.local in_tpkts = 26453 in_bytes = 1823777 out_tpkts = 96272 out_bytes = 880110475 exception_packets = 38 exception_packets_dropped = 8 exception_packets_allowed = 30 total_flows = 0 aged_flows = 0 uptime = 1765417400759689 phy_if_info= [ [ _iter254->first = eth0 [ speed = -1 duplexity = 255 ], ] ] raw_vhost_stats= [ name = vhost0 in_pkts = 26453 in_bytes = 1823777 out_pkts = 96272 out_bytes = 880110475 speed = -1 duplexity = -1 ] raw_vhost_drop_stats= [ ds_discard = 0 ds_pull = 0 ds_invalid_if = 0 ds_invalid_arp = 0 ds_trap_no_if = 0 ds_nowhere_to_go = 0 ds_flow_queue_limit_exceeded = 0 ds_flow_no_memory = 0 ds_flow_invalid_protocol = 0 ds_flow_nat_no_rflow = 0 ds_flow_action_drop = 0 ds_flow_action_invalid = 0 ds_flow_unusable = 0 ds_flow_table_full = 0 ds_interface_tx_discard = 0 ds_interface_drop = 32 ds_duplicated = 0 ds_push = 0 ds_ttl_exceeded = 0 ds_invalid_nh = 0 ds_invalid_label = 0 ds_invalid_protocol = 0 ds_interface_rx_discard = 0 ds_invalid_mcast_source = 0 ds_head_alloc_fail = 0 ds_pcow_fail = 0 ds_mcast_clone_fail = 0 ds_rewrite_fail = 0 ds_misc = 0 ds_invalid_packet = 0 ds_cksum_err = 0 ds_no_fmd = 0 ds_invalid_vnid = 0 ds_frag_err = 0 ds_invalid_source = 0 ds_mcast_df_bit = 0 ds_l2_no_route = 0 ds_vlan_fwd_tx = 0 ds_vlan_fwd_enq = 0 ds_drop_new_flow = 0 ds_flow_evict = 0 ds_trap_original = 0 ds_no_memory = 0 ds_fragment_queue_fail = 0 ds_no_frag_entry = 0 ds_icmp_error = 0 ds_clone_fail = 0 ds_drop_pkts = 32 ] raw_drop_stats= [ ds_discard = 0 ds_pull = 0 ds_invalid_if = 0 ds_invalid_arp = 0 ds_trap_no_if = 0 ds_nowhere_to_go = 0 ds_flow_queue_limit_exceeded = 0 ds_flow_no_memory = 0 ds_flow_invalid_protocol = 0 ds_flow_nat_no_rflow = 0 ds_flow_action_drop = 0 ds_flow_action_invalid = 0 ds_flow_unusable = 0 ds_flow_table_full = 0 ds_interface_tx_discard = 0 ds_interface_drop = 32 ds_duplicated = 0 ds_push = 0 ds_ttl_exceeded = 0 ds_invalid_nh = 0 ds_invalid_label = 0 ds_invalid_protocol = 0 ds_interface_rx_discard = 0 ds_invalid_mcast_source = 0 ds_head_alloc_fail = 0 ds_pcow_fail = 0 ds_mcast_clone_fail = 0 ds_rewrite_fail = 0 ds_misc = 0 ds_invalid_packet = 0 ds_cksum_err = 0 ds_no_fmd = 0 ds_invalid_vnid = 0 ds_frag_err = 0 ds_invalid_source = 0 ds_mcast_df_bit = 0 ds_l2_no_route = 0 ds_vlan_fwd_tx = 0 ds_vlan_fwd_enq = 0 ds_drop_new_flow = 0 ds_flow_evict = 0 ds_trap_original = 0 ds_no_memory = 0 ds_fragment_queue_fail = 0 ds_no_frag_entry = 0 ds_icmp_error = 0 ds_clone_fail = 0 ds_drop_pkts = 32 ] flow_rate= [ added_flows = 0 max_flow_adds_per_second = 0 min_flow_adds_per_second = 0 deleted_flows = 0 max_flow_deletes_per_second = 0 min_flow_deletes_per_second = 0 active_flows = 0 hold_flows = 0 ] raw_phy_if_stats= [ [ _iter262->first = eth0 [ in_pkts = 96234 in_bytes = 880108879 out_pkts = 26456 out_bytes = 1823903 ], ] ] raw_phy_if_drop_stats= [ [ _iter263->first = eth0 [ ds_discard = 0 ds_pull = 0 ds_invalid_if = 0 ds_invalid_arp = 0 ds_trap_no_if = 0 ds_nowhere_to_go = 0 ds_flow_queue_limit_exceeded = 0 ds_flow_no_memory = 0 ds_flow_invalid_protocol = 0 ds_flow_nat_no_rflow = 0 ds_flow_action_drop = 0 ds_flow_action_invalid = 0 ds_flow_unusable = 0 ds_flow_table_full = 0 ds_interface_tx_discard = 0 ds_interface_drop = 32 ds_duplicated = 0 ds_push = 0 ds_ttl_exceeded = 0 ds_invalid_nh = 0 ds_invalid_label = 0 ds_invalid_protocol = 0 ds_interface_rx_discard = 0 ds_invalid_mcast_source = 0 ds_head_alloc_fail = 0 ds_pcow_fail = 0 ds_mcast_clone_fail = 0 ds_rewrite_fail = 0 ds_misc = 0 ds_invalid_packet = 0 ds_cksum_err = 0 ds_no_fmd = 0 ds_invalid_vnid = 0 ds_frag_err = 0 ds_invalid_source = 0 ds_mcast_df_bit = 0 ds_l2_no_route = 0 ds_vlan_fwd_tx = 0 ds_vlan_fwd_enq = 0 ds_drop_new_flow = 0 ds_flow_evict = 0 ds_trap_original = 0 ds_no_memory = 0 ds_fragment_queue_fail = 0 ds_no_frag_entry = 0 ds_icmp_error = 0 ds_clone_fail = 0 ds_drop_pkts = 32 ], ] ] ] 2025-12-11 Thu 01:43:51:013.765 UTC cn-jenkins-deploy-platform-k8s-manifests-278-1 [Thread 139889531872768, Pid 560]: SANDESH: Send FAILED: 1765417431013509 [SYS_NOTICE]: VrouterControlStatsTrace: data= [ name = cn-jenkins-deploy-platform-k8s-manifests-278-1.cluster.local raw_xmpp_stats= [ [ _iter306->first = 10.0.0.22 [ reconnects = 0 in_msgs = 0 out_msgs = 0 ], ] ] raw_ifmap_stats= [ [ _iter307->first = link_delete_parse_errors _iter307->second = 0, _iter307->first = link_update_parse_errors _iter307->second = 0, _iter307->first = node_delete_parse_errors _iter307->second = 0, _iter307->first = node_update_parse_errors _iter307->second = 0, ] ] raw_rt_table_size= [ [ _iter308->first = default-domain:default-project:ip-fabric:__default__ [ inet4_unicast = 7 inet4_multicast = 0 evpn = 0 bridge = 5 inet6_unicast = 2 ], _iter308->first = default-domain:default-project:ip-fabric:ip-fabric [ inet4_unicast = 2 inet4_multicast = 0 evpn = 1 bridge = 2 inet6_unicast = 0 ], ] ] ] 2025-12-11 Thu 01:43:51:454.254 UTC cn-jenkins-deploy-platform-k8s-manifests-278-1 [Thread 139889510880768, Pid 560]: SANDESH: Send FAILED: 1765417431454133 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.0.0.22 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-12-11 Thu 01:43:51:454.583 UTC cn-jenkins-deploy-platform-k8s-manifests-278-1 [Thread 139889543049216, Pid 560]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.0.0.22 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-12-11 Thu 01:43:51:454.635 UTC cn-jenkins-deploy-platform-k8s-manifests-278-1 [Thread 139889519277568, Pid 560]: SANDESH: Send FAILED: 1765417431454604 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.0.0.22 state = Connect event = Connect Event local_port = 41523 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-12-11 Thu 01:43:51:455.550 UTC cn-jenkins-deploy-platform-k8s-manifests-278-1 [Thread 139889536071168, Pid 560]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-12-11 Thu 01:43:51:455.645 UTC cn-jenkins-deploy-platform-k8s-manifests-278-1 [Thread 139889536071168, Pid 560]: SANDESH: Send FAILED: 1765417431455575 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.0.0.22 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-12-11 Thu 01:43:51:455.659 UTC cn-jenkins-deploy-platform-k8s-manifests-278-1 [Thread 139889536071168, Pid 560]: SANDESH: Send FAILED: 1765417431455579 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.0.0.22 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-12-11 Thu 01:43:52:017.111 UTC cn-jenkins-deploy-platform-k8s-manifests-278-1 [Thread 139889506682368, Pid 560]: SANDESH: Send FAILED: 1765417432015559 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-k8s-manifests-278-1.cluster.local process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.0.0.22 server_addrs= [ [ (*_iter6) = 10.0.0.22:5269, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.0.0.22 server_addrs= [ [ (*_iter6) = 10.0.0.22:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.22:8086, ] ] status = Initializing description = Idle : EvIdleHoldTimerExpired -> Connect ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.0.0.22, XMPP:dns-server:10.0.0.22, Collector connection down, No Configuration for self ], ] ] ] 2025-12-11 Thu 01:43:52:017.210 UTC cn-jenkins-deploy-platform-k8s-manifests-278-1 [Thread 139889506682368, Pid 560]: TCP [SYS_ERR]: TcpSessionMessageLog: Session -(-1) > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-12-11 Thu 01:43:52:017.541 UTC cn-jenkins-deploy-platform-k8s-manifests-278-1 [Thread 139889506682368, Pid 560]: SANDESH: Send FAILED: 1765417432017059 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-k8s-manifests-278-1.cluster.local:Compute:contrail-vrouter-agent:0 client_info= [ status = Connect successful_connections = 0 pid = 560 http_port = 8085 start_time = 1765417401006781 collector_name = collector_ip = 10.0.0.22:8086 collector_list= [ [ (*_iter40) = 10.0.0.22:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 15 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 14 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 29 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterControlStatsTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 44 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 12 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 44, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 80, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-12-11 Thu 01:43:52:017.592 UTC cn-jenkins-deploy-platform-k8s-manifests-278-1 [Thread 139889506682368, Pid 560]: SANDESH: Send FAILED: 1765417432017187 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-k8s-manifests-278-1.cluster.local process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.0.0.22 server_addrs= [ [ (*_iter6) = 10.0.0.22:5269, ] ] status = Down description = Connect ], [ type = XMPP name = dns-server:10.0.0.22 server_addrs= [ [ (*_iter6) = 10.0.0.22:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.0.0.22, XMPP:dns-server:10.0.0.22, Collector connection down, No Configuration for self ], ] ] ] 2025-12-11 Thu 01:43:52:017.823 UTC cn-jenkins-deploy-platform-k8s-manifests-278-1 [Thread 139889506682368, Pid 560]: SANDESH: Send FAILED: 1765417432017291 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-k8s-manifests-278-1.cluster.local:Compute:contrail-vrouter-agent:0 client_info= [ status = Idle successful_connections = 0 pid = 560 http_port = 8085 start_time = 1765417401006781 collector_name = collector_ip = 10.0.0.22:8086 collector_list= [ [ (*_iter40) = 10.0.0.22:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 16 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 14 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 30 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterControlStatsTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 44 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 12 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 45, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 81, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-12-11 Thu 01:44:18:693.979 UTC cn-jenkins-deploy-platform-k8s-manifests-278-1 [Thread 139889536071168, Pid 560]: SANDESH: Send FAILED: 1765417458693872 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.0.0.22 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-12-11 Thu 01:44:18:694.651 UTC cn-jenkins-deploy-platform-k8s-manifests-278-1 [Thread 139889543049216, Pid 560]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer RECV Mode Client: Event: Tcp Connect Fail peer ip: 10.0.0.22 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1299 2025-12-11 Thu 01:44:18:694.714 UTC cn-jenkins-deploy-platform-k8s-manifests-278-1 [Thread 139889531872768, Pid 560]: SANDESH: Send FAILED: 1765417458694679 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.0.0.22 state = Connect event = Connect Event local_port = 44417 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-12-11 Thu 01:44:18:695.280 UTC cn-jenkins-deploy-platform-k8s-manifests-278-1 [Thread 139889515079168, Pid 560]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-12-11 Thu 01:44:18:695.346 UTC cn-jenkins-deploy-platform-k8s-manifests-278-1 [Thread 139889510880768, Pid 560]: SANDESH: Send FAILED: 1765417458695332 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.0.0.22 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-12-11 Thu 01:44:18:695.443 UTC cn-jenkins-deploy-platform-k8s-manifests-278-1 [Thread 139889510880768, Pid 560]: SANDESH: Send FAILED: 1765417458695343 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.0.0.22 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-12-11 Thu 01:44:18:996.374 UTC cn-jenkins-deploy-platform-k8s-manifests-278-1 [Thread 139889510880768, Pid 560]: SANDESH: Send FAILED: 1765417458995756 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.0.0.22 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-12-11 Thu 01:44:18:996.508 UTC cn-jenkins-deploy-platform-k8s-manifests-278-1 [Thread 139889510880768, Pid 560]: SANDESH: Send FAILED: 1765417458995953 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.0.0.22 state = Connect event = Connect Event local_port = 33585 remote_port = 5269 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-12-11 Thu 01:44:19:003.365 UTC cn-jenkins-deploy-platform-k8s-manifests-278-1 [Thread 139889543049216, Pid 560]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer cn-jenkins-deploy-platform-k8s-manifests-278-1.cluster.local:10.0.0.22 RECV Mode Client: Event: Tcp Connected peer ip: 10.0.0.22 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1292 2025-12-11 Thu 01:44:19:004.244 UTC cn-jenkins-deploy-platform-k8s-manifests-278-1 [Thread 139889510880768, Pid 560]: SANDESH: Send FAILED: 1765417459004203 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.0.0.22 state = Connect next_state = OpenSent event = EvTcpConnected local_port = 33585 remote_port = 5269 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-12-11 Thu 01:44:19:015.887 UTC cn-jenkins-deploy-platform-k8s-manifests-278-1 [Thread 139889519277568, Pid 560]: SANDESH: Send FAILED: 1765417459015834 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.0.0.22 state = OpenSent next_state = Established event = EvXmppOpen identifier = cn-jenkins-deploy-platform-k8s-manifests-278-1.cluster.local ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-12-11 Thu 01:44:19:017.095 UTC cn-jenkins-deploy-platform-k8s-manifests-278-1 [Thread 139889519277568, Pid 560]: SANDESH: Send FAILED: 1765417459015948 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-k8s-manifests-278-1.cluster.local process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.0.0.22 server_addrs= [ [ (*_iter6) = 10.0.0.22:5269, ] ] status = Up description = OpenSent ], [ type = XMPP name = dns-server:10.0.0.22 server_addrs= [ [ (*_iter6) = 10.0.0.22:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:dns-server:10.0.0.22, Collector connection down, No Configuration for self ], ] ] ] 2025-12-11 Thu 01:44:19:017.209 UTC cn-jenkins-deploy-platform-k8s-manifests-278-1 [Thread 139889543049216, Pid 560]: TCP [SYS_ERR]: TcpSessionMessageLog: Session 10.0.0.22:33585::10.0.0.22:5269 > Write failed due to error: Broken pipe src/contrail-common/io/tcp_session.cc 384 2025-12-11 Thu 01:44:19:017.226 UTC cn-jenkins-deploy-platform-k8s-manifests-278-1 [Thread 139889543049216, Pid 560]: TCP [SYS_ERR]: TcpSessionMessageLog: Session 10.0.0.22:33585::10.0.0.22:5269 > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-12-11 Thu 01:44:19:017.248 UTC cn-jenkins-deploy-platform-k8s-manifests-278-1 [Thread 139889543049216, Pid 560]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer cn-jenkins-deploy-platform-k8s-manifests-278-1.cluster.local:10.0.0.22 RECV Mode Client: Event: Tcp Connection Closed peer ip: 10.0.0.22 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1307 2025-12-11 Thu 01:44:19:018.238 UTC cn-jenkins-deploy-platform-k8s-manifests-278-1 [Thread 139889527674368, Pid 560]: SANDESH: Send FAILED: 1765417459018101 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.0.0.22 state = Established next_state = Active event = EvTcpClose ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-12-11 Thu 01:44:19:018.257 UTC cn-jenkins-deploy-platform-k8s-manifests-278-1 [Thread 139889527674368, Pid 560]: SANDESH: Send FAILED: 1765417459018106 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.0.0.22 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-12-11 Thu 01:44:19:018.275 UTC cn-jenkins-deploy-platform-k8s-manifests-278-1 [Thread 139889527674368, Pid 560]: SANDESH: Send FAILED: 1765417459018156 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-k8s-manifests-278-1.cluster.local process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.0.0.22 server_addrs= [ [ (*_iter6) = 10.0.0.22:5269, ] ] status = Down description = Established ], [ type = XMPP name = dns-server:10.0.0.22 server_addrs= [ [ (*_iter6) = 10.0.0.22:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.0.0.22, XMPP:dns-server:10.0.0.22, Collector connection down, No Configuration for self ], ] ] ] 2025-12-11 Thu 01:44:21:009.311 UTC cn-jenkins-deploy-platform-k8s-manifests-278-1 [Thread 139889531872768, Pid 560]: SANDESH: Send FAILED: 1765417461009153 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-k8s-manifests-278-1.cluster.local:Compute:contrail-vrouter-agent:0 sm_queue_count = 0 sm_stats= [ ev_stats= [ [ [ event = scm::EvCollectorUpdate enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvIdleHoldTimerExpired enqueues = 5 dequeues = 5 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvSandeshSend enqueues = 96 dequeues = 96 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvStart enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpConnectFail enqueues = 6 dequeues = 6 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpDeleteSession enqueues = 6 dequeues = 6 enqueue_fails = 0 dequeue_fails = 0 ], [ event = enqueues = 115 dequeues = 115 enqueue_fails = 0 dequeue_fails = 0 ], ] ] state = Idle state_since = 1765417432017248 last_state = Idle last_event = scm::EvSandeshSend last_event_at = 1765417459018263 ] max_sm_queue_count = 4 ] 2025-12-11 Thu 01:44:21:009.454 UTC cn-jenkins-deploy-platform-k8s-manifests-278-1 [Thread 139889531872768, Pid 560]: SANDESH: Send FAILED: 1765417461009259 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-k8s-manifests-278-1.cluster.local:Compute:contrail-vrouter-agent:0 client_info= [ status = Idle successful_connections = 0 pid = 560 http_port = 8085 start_time = 1765417401006781 collector_name = collector_ip = 10.0.0.22:8086 collector_list= [ [ (*_iter40) = 10.0.0.22:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 19 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 16 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 46 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterControlStatsTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 54 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 15 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 64, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 96, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-12-11 Thu 01:44:21:013.946 UTC cn-jenkins-deploy-platform-k8s-manifests-278-1 [Thread 139889536071168, Pid 560]: SANDESH: Send FAILED: 1765417461013520 [SYS_NOTICE]: UveVrouterAgent: data= [ name = cn-jenkins-deploy-platform-k8s-manifests-278-1.cluster.local xmpp_peer_list= [ [ [ ip = 10.0.0.22 status = 0 setup_time = 1765417459015956 primary = 0 ], ] ] ] 2025-12-11 Thu 01:44:21:013.997 UTC cn-jenkins-deploy-platform-k8s-manifests-278-1 [Thread 139889536071168, Pid 560]: SANDESH: Send FAILED: 1765417461013846 [SYS_NOTICE]: VrouterStats: data= [ name = cn-jenkins-deploy-platform-k8s-manifests-278-1.cluster.local cpu_info= [ num_cpu = 4 sys_mem_info= [ total = 2451227232 used = 32570 free = 0 buffers = 0 cached = 2451227136 node_type = ] meminfo= [ virt = 4294967295 peakvirt = 18266505 res = 0 ] cpuload= [ one_min_avg = 1.3075 five_min_avg = 0.4575 fifteen_min_avg = 0.18 ] cpu_share = 0.706919 ] ] 2025-12-11 Thu 01:44:21:014.012 UTC cn-jenkins-deploy-platform-k8s-manifests-278-1 [Thread 139889536071168, Pid 560]: SANDESH: Send FAILED: 1765417461013870 [SYS_NOTICE]: ComputeCpuStateTrace: data= [ name = cn-jenkins-deploy-platform-k8s-manifests-278-1.cluster.local cpu_info= [ [ [ mem_virt = 4294967295 cpu_share = 0.706919 used_sys_mem = 1843772730 one_min_cpuload = 1.3075 mem_res = 0 ], ] ] ] 2025-12-11 Thu 01:44:21:014.118 UTC cn-jenkins-deploy-platform-k8s-manifests-278-1 [Thread 139889536071168, Pid 560]: SANDESH: Send FAILED: 1765417461013894 [SYS_NOTICE]: VrouterStats: data= [ name = cn-jenkins-deploy-platform-k8s-manifests-278-1.cluster.local in_tpkts = 39973 in_bytes = 2796576 out_tpkts = 139798 out_bytes = 1285037012 exception_packets = 40 exception_packets_dropped = 9 exception_packets_allowed = 31 raw_vhost_stats= [ name = vhost0 in_pkts = 39973 in_bytes = 2796576 out_pkts = 139798 out_bytes = 1285037012 speed = -1 duplexity = -1 ] raw_vhost_drop_stats= [ ds_discard = 0 ds_pull = 0 ds_invalid_if = 0 ds_invalid_arp = 0 ds_trap_no_if = 0 ds_nowhere_to_go = 0 ds_flow_queue_limit_exceeded = 0 ds_flow_no_memory = 0 ds_flow_invalid_protocol = 0 ds_flow_nat_no_rflow = 0 ds_flow_action_drop = 0 ds_flow_action_invalid = 0 ds_flow_unusable = 0 ds_flow_table_full = 0 ds_interface_tx_discard = 0 ds_interface_drop = 32 ds_duplicated = 0 ds_push = 0 ds_ttl_exceeded = 0 ds_invalid_nh = 0 ds_invalid_label = 0 ds_invalid_protocol = 0 ds_interface_rx_discard = 0 ds_invalid_mcast_source = 0 ds_head_alloc_fail = 0 ds_pcow_fail = 0 ds_mcast_clone_fail = 0 ds_rewrite_fail = 0 ds_misc = 0 ds_invalid_packet = 0 ds_cksum_err = 0 ds_no_fmd = 0 ds_invalid_vnid = 0 ds_frag_err = 0 ds_invalid_source = 0 ds_mcast_df_bit = 0 ds_l2_no_route = 0 ds_vlan_fwd_tx = 0 ds_vlan_fwd_enq = 0 ds_drop_new_flow = 0 ds_flow_evict = 0 ds_trap_original = 0 ds_no_memory = 0 ds_fragment_queue_fail = 0 ds_no_frag_entry = 0 ds_icmp_error = 0 ds_clone_fail = 0 ds_drop_pkts = 32 ] raw_drop_stats= [ ds_discard = 0 ds_pull = 0 ds_invalid_if = 0 ds_invalid_arp = 0 ds_trap_no_if = 0 ds_nowhere_to_go = 0 ds_flow_queue_limit_exceeded = 0 ds_flow_no_memory = 0 ds_flow_invalid_protocol = 0 ds_flow_nat_no_rflow = 0 ds_flow_action_drop = 0 ds_flow_action_invalid = 0 ds_flow_unusable = 0 ds_flow_table_full = 0 ds_interface_tx_discard = 0 ds_interface_drop = 32 ds_duplicated = 0 ds_push = 0 ds_ttl_exceeded = 0 ds_invalid_nh = 0 ds_invalid_label = 0 ds_invalid_protocol = 0 ds_interface_rx_discard = 0 ds_invalid_mcast_source = 0 ds_head_alloc_fail = 0 ds_pcow_fail = 0 ds_mcast_clone_fail = 0 ds_rewrite_fail = 0 ds_misc = 0 ds_invalid_packet = 0 ds_cksum_err = 0 ds_no_fmd = 0 ds_invalid_vnid = 0 ds_frag_err = 0 ds_invalid_source = 0 ds_mcast_df_bit = 0 ds_l2_no_route = 0 ds_vlan_fwd_tx = 0 ds_vlan_fwd_enq = 0 ds_drop_new_flow = 0 ds_flow_evict = 0 ds_trap_original = 0 ds_no_memory = 0 ds_fragment_queue_fail = 0 ds_no_frag_entry = 0 ds_icmp_error = 0 ds_clone_fail = 0 ds_drop_pkts = 32 ] flow_rate= [ added_flows = 0 max_flow_adds_per_second = 0 min_flow_adds_per_second = 0 deleted_flows = 0 max_flow_deletes_per_second = 0 min_flow_deletes_per_second = 0 active_flows = 0 hold_flows = 0 ] raw_phy_if_stats= [ [ _iter262->first = eth0 [ in_pkts = 139745 in_bytes = 1285034786 out_pkts = 39976 out_bytes = 2796702 ], ] ] raw_phy_if_drop_stats= [ [ _iter263->first = eth0 [ ds_discard = 0 ds_pull = 0 ds_invalid_if = 0 ds_invalid_arp = 0 ds_trap_no_if = 0 ds_nowhere_to_go = 0 ds_flow_queue_limit_exceeded = 0 ds_flow_no_memory = 0 ds_flow_invalid_protocol = 0 ds_flow_nat_no_rflow = 0 ds_flow_action_drop = 0 ds_flow_action_invalid = 0 ds_flow_unusable = 0 ds_flow_table_full = 0 ds_interface_tx_discard = 0 ds_interface_drop = 32 ds_duplicated = 0 ds_push = 0 ds_ttl_exceeded = 0 ds_invalid_nh = 0 ds_invalid_label = 0 ds_invalid_protocol = 0 ds_interface_rx_discard = 0 ds_invalid_mcast_source = 0 ds_head_alloc_fail = 0 ds_pcow_fail = 0 ds_mcast_clone_fail = 0 ds_rewrite_fail = 0 ds_misc = 0 ds_invalid_packet = 0 ds_cksum_err = 0 ds_no_fmd = 0 ds_invalid_vnid = 0 ds_frag_err = 0 ds_invalid_source = 0 ds_mcast_df_bit = 0 ds_l2_no_route = 0 ds_vlan_fwd_tx = 0 ds_vlan_fwd_enq = 0 ds_drop_new_flow = 0 ds_flow_evict = 0 ds_trap_original = 0 ds_no_memory = 0 ds_fragment_queue_fail = 0 ds_no_frag_entry = 0 ds_icmp_error = 0 ds_clone_fail = 0 ds_drop_pkts = 32 ], ] ] ] 2025-12-11 Thu 01:44:21:014.148 UTC cn-jenkins-deploy-platform-k8s-manifests-278-1 [Thread 139889536071168, Pid 560]: SANDESH: Send FAILED: 1765417461013920 [SYS_NOTICE]: VrouterControlStatsTrace: data= [ name = cn-jenkins-deploy-platform-k8s-manifests-278-1.cluster.local raw_xmpp_stats= [ [ _iter306->first = 10.0.0.22 [ reconnects = 1 in_msgs = 0 out_msgs = 1 ], ] ] raw_ifmap_stats= [ [ _iter307->first = link_delete_parse_errors _iter307->second = 0, _iter307->first = link_update_parse_errors _iter307->second = 0, _iter307->first = node_delete_parse_errors _iter307->second = 0, _iter307->first = node_update_parse_errors _iter307->second = 0, ] ] raw_rt_table_size= [ [ _iter308->first = default-domain:default-project:ip-fabric:__default__ [ inet4_unicast = 7 inet4_multicast = 0 evpn = 0 bridge = 5 inet6_unicast = 2 ], _iter308->first = default-domain:default-project:ip-fabric:ip-fabric [ inet4_unicast = 2 inet4_multicast = 0 evpn = 1 bridge = 2 inet6_unicast = 0 ], ] ] ] 2025-12-11 Thu 01:44:22:021.552 UTC cn-jenkins-deploy-platform-k8s-manifests-278-1 [Thread 139889536071168, Pid 560]: SANDESH: Send FAILED: 1765417462020156 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-k8s-manifests-278-1.cluster.local process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.0.0.22 server_addrs= [ [ (*_iter6) = 10.0.0.22:5269, ] ] status = Down description = Established ], [ type = XMPP name = dns-server:10.0.0.22 server_addrs= [ [ (*_iter6) = 10.0.0.22:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.22:8086, ] ] status = Initializing description = Idle : EvIdleHoldTimerExpired -> Connect ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.0.0.22, XMPP:dns-server:10.0.0.22, Collector connection down, No Configuration for self ], ] ] ] 2025-12-11 Thu 01:44:22:021.626 UTC cn-jenkins-deploy-platform-k8s-manifests-278-1 [Thread 139889536071168, Pid 560]: TCP [SYS_ERR]: TcpSessionMessageLog: Session -(-1) > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 313 2025-12-11 Thu 01:44:22:021.826 UTC cn-jenkins-deploy-platform-k8s-manifests-278-1 [Thread 139889536071168, Pid 560]: SANDESH: Send FAILED: 1765417462021516 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-k8s-manifests-278-1.cluster.local:Compute:contrail-vrouter-agent:0 client_info= [ status = Connect successful_connections = 0 pid = 560 http_port = 8085 start_time = 1765417401006781 collector_name = collector_ip = 10.0.0.22:8086 collector_list= [ [ (*_iter40) = 10.0.0.22:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ComputeCpuStateTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 19 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 18 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 46 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 3 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterControlStatsTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 3 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 54 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 15 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 64, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 103, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-12-11 Thu 01:44:22:021.865 UTC cn-jenkins-deploy-platform-k8s-manifests-278-1 [Thread 139889536071168, Pid 560]: SANDESH: Send FAILED: 1765417462021607 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-k8s-manifests-278-1.cluster.local process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.0.0.22 server_addrs= [ [ (*_iter6) = 10.0.0.22:5269, ] ] status = Down description = Established ], [ type = XMPP name = dns-server:10.0.0.22 server_addrs= [ [ (*_iter6) = 10.0.0.22:53, ] ] status = Down description = Connect ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.0.0.22, XMPP:dns-server:10.0.0.22, Collector connection down, No Configuration for self ], ] ] ] 2025-12-11 Thu 01:44:22:021.993 UTC cn-jenkins-deploy-platform-k8s-manifests-278-1 [Thread 139889536071168, Pid 560]: SANDESH: Send FAILED: 1765417462021683 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-k8s-manifests-278-1.cluster.local:Compute:contrail-vrouter-agent:0 client_info= [ status = Idle successful_connections = 0 pid = 560 http_port = 8085 start_time = 1765417401006781 collector_name = collector_ip = 10.0.0.22:8086 collector_list= [ [ (*_iter40) = 10.0.0.22:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ComputeCpuStateTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 20 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 18 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 47 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 3 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterControlStatsTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 3 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 54 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 15 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 65, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 104, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-12-11 Thu 01:44:46:236.199 UTC cn-jenkins-deploy-platform-k8s-manifests-278-1 [Thread 139889519277568, Pid 560]: SANDESH: Send FAILED: 1765417486236023 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.0.0.22 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-12-11 Thu 01:44:46:239.288 UTC cn-jenkins-deploy-platform-k8s-manifests-278-1 [Thread 139889543049216, Pid 560]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer cn-jenkins-deploy-platform-k8s-manifests-278-1.cluster.local:10.0.0.22 RECV Mode Client: Event: Tcp Connected peer ip: 10.0.0.22 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1292 2025-12-11 Thu 01:44:46:239.465 UTC cn-jenkins-deploy-platform-k8s-manifests-278-1 [Thread 139889506682368, Pid 560]: SANDESH: Send FAILED: 1765417486239340 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.0.0.22 state = Connect event = Connect Event local_port = 37735 remote_port = 8093 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-12-11 Thu 01:44:46:239.485 UTC cn-jenkins-deploy-platform-k8s-manifests-278-1 [Thread 139889506682368, Pid 560]: SANDESH: Send FAILED: 1765417486239444 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.0.0.22 state = Connect next_state = OpenSent event = EvTcpConnected local_port = 37735 remote_port = 8093 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-12-11 Thu 01:44:46:259.245 UTC cn-jenkins-deploy-platform-k8s-manifests-278-1 [Thread 139889510880768, Pid 560]: SANDESH: Send FAILED: 1765417486259197 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.0.0.22 state = OpenSent next_state = Established event = EvXmppOpen identifier = cn-jenkins-deploy-platform-k8s-manifests-278-1.cluster.local ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-12-11 Thu 01:44:46:259.669 UTC cn-jenkins-deploy-platform-k8s-manifests-278-1 [Thread 139889506682368, Pid 560]: SANDESH: Send FAILED: 1765417486259630 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-k8s-manifests-278-1.cluster.local process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.0.0.22 server_addrs= [ [ (*_iter6) = 10.0.0.22:5269, ] ] status = Down description = Established ], [ type = XMPP name = dns-server:10.0.0.22 server_addrs= [ [ (*_iter6) = 10.0.0.22:53, ] ] status = Up description = OpenSent ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = XMPP:control-node:10.0.0.22, Collector connection down, No Configuration for self ], ] ] ] 2025-12-11 Thu 01:44:48:448.809 UTC cn-jenkins-deploy-platform-k8s-manifests-278-1 [Thread 139889531872768, Pid 560]: SANDESH: Send FAILED: 1765417488448663 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.0.0.22 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-12-11 Thu 01:44:48:451.296 UTC cn-jenkins-deploy-platform-k8s-manifests-278-1 [Thread 139889543049216, Pid 560]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer cn-jenkins-deploy-platform-k8s-manifests-278-1.cluster.local:10.0.0.22 RECV Mode Client: Event: Tcp Connected peer ip: 10.0.0.22 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1292 2025-12-11 Thu 01:44:48:451.452 UTC cn-jenkins-deploy-platform-k8s-manifests-278-1 [Thread 139889527674368, Pid 560]: SANDESH: Send FAILED: 1765417488451342 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.0.0.22 state = Connect event = Connect Event local_port = 38687 remote_port = 5269 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-12-11 Thu 01:44:48:451.470 UTC cn-jenkins-deploy-platform-k8s-manifests-278-1 [Thread 139889527674368, Pid 560]: SANDESH: Send FAILED: 1765417488451436 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.0.0.22 state = Connect next_state = OpenSent event = EvTcpConnected local_port = 38687 remote_port = 5269 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-12-11 Thu 01:44:48:452.449 UTC cn-jenkins-deploy-platform-k8s-manifests-278-1 [Thread 139889531872768, Pid 560]: SANDESH: Send FAILED: 1765417488452426 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.0.0.22 state = OpenSent next_state = Established event = EvXmppOpen identifier = cn-jenkins-deploy-platform-k8s-manifests-278-1.cluster.local ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-12-11 Thu 01:44:48:452.571 UTC cn-jenkins-deploy-platform-k8s-manifests-278-1 [Thread 139889531872768, Pid 560]: SANDESH: Send FAILED: 1765417488452498 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-k8s-manifests-278-1.cluster.local process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = XMPP name = control-node:10.0.0.22 server_addrs= [ [ (*_iter6) = 10.0.0.22:5269, ] ] status = Up description = OpenSent ], [ type = XMPP name = dns-server:10.0.0.22 server_addrs= [ [ (*_iter6) = 10.0.0.22:53, ] ] status = Up description = OpenSent ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = Collector connection down, No Configuration for self ], ] ] ] 2025-12-11 Thu 01:44:48:507.972 UTC cn-jenkins-deploy-platform-k8s-manifests-278-1 [Thread 139889531872768, Pid 560]: SANDESH: Send FAILED: 1765417488507847 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-k8s-manifests-278-1.cluster.local process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Functional connection_infos= [ [ [ type = XMPP name = control-node:10.0.0.22 server_addrs= [ [ (*_iter6) = 10.0.0.22:5269, ] ] status = Up description = OpenSent ], [ type = XMPP name = dns-server:10.0.0.22 server_addrs= [ [ (*_iter6) = 10.0.0.22:53, ] ] status = Up description = OpenSent ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = Collector connection down ], ] ] ] 2025-12-11 Thu 01:44:48:560.199 UTC cn-jenkins-deploy-platform-k8s-manifests-278-1 [Thread 139889531872768, Pid 560]: SANDESH: Send FAILED: 1765417488560101 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-k8s-manifests-278-1.cluster.local process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Functional connection_infos= [ [ [ type = XMPP name = control-node:10.0.0.22 server_addrs= [ [ (*_iter6) = 10.0.0.22:5269, ] ] status = Up description = OpenSent ], [ type = XMPP name = dns-server:10.0.0.22 server_addrs= [ [ (*_iter6) = 10.0.0.22:53, ] ] status = Up description = OpenSent ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], ] ] flag_infos= [ [ ] ] description = Collector connection down ], ] ] ] 2025-12-11 Thu 01:44:48:561.251 UTC cn-jenkins-deploy-platform-k8s-manifests-278-1 [Thread 139889527674368, Pid 560]: SANDESH: Send FAILED: 1765417488560616 AgentVn [SYS_INFO]: VnObjectLog: vn= [ event = Addition uuid = 028323e6-dab0-4cb1-ae21-a70a79579b0a name = default-domain:default-project:ip-fabric logical_router_uuid = 00000000-0000-0000-0000-000000000000 vrf = default-domain:default-project:ip-fabric:ip-fabric ipv4_forwarding = 1 bridging = 1 admin_state = 1 vxlan_routing_vn = 0 oper_vxlan_id = 2 ] file = controller/src/vnsw/agent/oper/vn.cc line = 1408 2025-12-11 Thu 01:44:48:572.904 UTC cn-jenkins-deploy-platform-k8s-manifests-278-1 [Thread 139889531872768, Pid 560]: SANDESH: Send FAILED: 1765417488571735 AgentVn [SYS_INFO]: VnObjectLog: vn= [ event = Modification uuid = 028323e6-dab0-4cb1-ae21-a70a79579b0a name = default-domain:default-project:ip-fabric logical_router_uuid = 00000000-0000-0000-0000-000000000000 vrf = default-domain:default-project:ip-fabric:ip-fabric ipv4_forwarding = 1 bridging = 1 admin_state = 1 vxlan_routing_vn = 0 oper_vxlan_id = 2 ] file = controller/src/vnsw/agent/oper/vn.cc line = 1408 2025-12-11 Thu 01:44:50:762.611 UTC cn-jenkins-deploy-platform-k8s-manifests-278-1 [Thread 139889531872768, Pid 560]: SANDESH: Send FAILED: 1765417490762563 [SYS_NOTICE]: UveVirtualNetworkAgentTrace: data= [ name = default-domain:default-project:ip-fabric total_acl_rules = 0 interface_list= [ [ (*_iter66) = default-global-system-config:cn-jenkins-deploy-platform-k8s-manifests-278-1.cluster.local:vhost0, ] ] virtualmachine_list= [ [ ] ] acl = mirror_acl = ] 2025-12-11 Thu 01:44:50:762.878 UTC cn-jenkins-deploy-platform-k8s-manifests-278-1 [Thread 139889527674368, Pid 560]: SANDESH: Send FAILED: 1765417490762677 [SYS_NOTICE]: UveVMInterfaceAgentTrace: data= [ name = default-global-system-config:cn-jenkins-deploy-platform-k8s-manifests-278-1.cluster.local:vhost0 ip_address = 10.0.0.22 virtual_network = default-domain:default-project:ip-fabric mac_address = fa:16:3e:5b:8e:b3 label = 16 active = 1 vm_name = l2_active = 1 uuid = 14e9eb66-8830-467a-8205-bf2c183d859d ip6_address = :: ip6_active = 1 floating_ips= [ [ ] ] vm_uuid = ip4_active = 1 is_health_check_active = 1 health_check_instance_list= [ [ ] ] admin_state = 1 fixed_ip4_list= [ [ (*_iter26) = 10.0.0.22, ] ] fixed_ip6_list= [ [ ] ] vn_uuid = 028323e6-dab0-4cb1-ae21-a70a79579b0a tx_vlan = 65535 rx_vlan = 65535 parent_interface = eth0 vhostuser_mode = 0 port_mirror_enabled = 0 ] 2025-12-11 Thu 01:44:50:762.898 UTC cn-jenkins-deploy-platform-k8s-manifests-278-1 [Thread 139889527674368, Pid 560]: SANDESH: Send FAILED: 1765417490762693 [SYS_NOTICE]: UVEVMITags: data= [ name = default-global-system-config:cn-jenkins-deploy-platform-k8s-manifests-278-1.cluster.local:vhost0 app = tier = site = deployment = labels= [ [ ] ] custom_tags= [ [ ] ] ] 2025-12-11 Thu 01:44:50:764.562 UTC cn-jenkins-deploy-platform-k8s-manifests-278-1 [Thread 139889531872768, Pid 560]: SANDESH: Send FAILED: 1765417490764436 [SYS_NOTICE]: UveVirtualNetworkAgentTrace: data= [ name = default-domain:default-project:ip-fabric associated_fip_count = 0 in_bandwidth_usage = 0 out_bandwidth_usage = 0 vrf_stats_list= [ [ [ name = default-domain:default-project:ip-fabric:ip-fabric diag_packet_count = 0 unknown_unicast_floods = 0 arp_packet_counts= [ from_vm_interface= [ stats= [ proxies = 0 stitches = 0 floods = 0 ] ] from_physical_interface= [ stats= [ proxies = 0 stitches = 0 floods = 0 ] ] ] offload_packet_counts= [ gro = 0 ] nh_packet_counts= [ discards = 0 resolves = 0 l3_receives = 0 l2_receives = 0 local_vm_l3_forwards = 0 local_vm_l2_forwards = 0 vrf_translates = 0 ecmp_forwards = 0 tunnel_nh_stats= [ udp_encaps = 0 mpls_over_udp_encaps = 0 mpls_over_gre_encaps = 0 vxlan_encaps = 0 ] comp_nh_stats= [ source_replication_forwards = 0 edge_replication_forwards = 0 local_vm_l3_forwards = 0 total_multicast_forwards = 0 ] ] ], ] ] ingress_flow_count = 0 egress_flow_count = 0 ] 2025-12-11 Thu 01:44:50:764.648 UTC cn-jenkins-deploy-platform-k8s-manifests-278-1 [Thread 139889531872768, Pid 560]: SANDESH: Send FAILED: 1765417490764488 [SYS_NOTICE]: UVEVMIStats: data= [ name = default-global-system-config:cn-jenkins-deploy-platform-k8s-manifests-278-1.cluster.local:vhost0 vm_name = vm_uuid = virtual_network = default-domain:default-project:ip-fabric port_bucket_bmap= [ udp_sport_bitmap= [ [ (*_iter24) = 0, (*_iter24) = 0, (*_iter24) = 0, (*_iter24) = 0, (*_iter24) = 0, (*_iter24) = 0, (*_iter24) = 0, (*_iter24) = 0, ] ] udp_dport_bitmap= [ [ (*_iter25) = 0, (*_iter25) = 0, (*_iter25) = 0, (*_iter25) = 0, (*_iter25) = 0, (*_iter25) = 0, (*_iter25) = 0, (*_iter25) = 0, ] ] tcp_sport_bitmap= [ [ (*_iter26) = 0, (*_iter26) = 0, (*_iter26) = 0, (*_iter26) = 0, (*_iter26) = 0, (*_iter26) = 0, (*_iter26) = 0, (*_iter26) = 0, ] ] tcp_dport_bitmap= [ [ (*_iter27) = 0, (*_iter27) = 0, (*_iter27) = 0, (*_iter27) = 0, (*_iter27) = 0, (*_iter27) = 0, (*_iter27) = 0, (*_iter27) = 0, ] ] ] raw_if_stats= [ in_pkts = 45170 in_bytes = 3177966 out_pkts = 158641 out_bytes = 1492823761 ] flow_rate= [ added_flows = 0 max_flow_adds_per_second = 0 min_flow_adds_per_second = 0 deleted_flows = 0 max_flow_deletes_per_second = 0 min_flow_deletes_per_second = 0 active_flows = 0 hold_flows = 0 ] in_bw_usage = 0 out_bw_usage = 0 raw_drop_stats= [ ds_discard = 0 ds_pull = 0 ds_invalid_if = 0 ds_invalid_arp = 0 ds_trap_no_if = 0 ds_nowhere_to_go = 0 ds_flow_queue_limit_exceeded = 0 ds_flow_no_memory = 0 ds_flow_invalid_protocol = 0 ds_flow_nat_no_rflow = 0 ds_flow_action_drop = 0 ds_flow_action_invalid = 0 ds_flow_unusable = 0 ds_flow_table_full = 0 ds_interface_tx_discard = 0 ds_interface_drop = 32 ds_duplicated = 0 ds_push = 0 ds_ttl_exceeded = 0 ds_invalid_nh = 0 ds_invalid_label = 0 ds_invalid_protocol = 0 ds_interface_rx_discard = 0 ds_invalid_mcast_source = 0 ds_head_alloc_fail = 0 ds_pcow_fail = 0 ds_mcast_clone_fail = 0 ds_rewrite_fail = 0 ds_misc = 0 ds_invalid_packet = 0 ds_cksum_err = 0 ds_no_fmd = 0 ds_invalid_vnid = 0 ds_frag_err = 0 ds_invalid_source = 0 ds_mcast_df_bit = 0 ds_l2_no_route = 0 ds_vlan_fwd_tx = 0 ds_vlan_fwd_enq = 0 ds_drop_new_flow = 0 ds_flow_evict = 0 ds_trap_original = 0 ds_no_memory = 0 ds_fragment_queue_fail = 0 ds_no_frag_entry = 0 ds_icmp_error = 0 ds_clone_fail = 0 ds_drop_pkts = 32 ] policy_rules= [ [ ] ] ] 2025-12-11 Thu 01:44:50:764.662 UTC cn-jenkins-deploy-platform-k8s-manifests-278-1 [Thread 139889531872768, Pid 560]: SANDESH: Send FAILED: 1765417490764491 [SYS_INFO]: EndpointSecurityStats: name = default-global-system-config:cn-jenkins-deploy-platform-k8s-manifests-278-1.cluster.local:vhost0 eps= [ [ ] ] file = controller/src/vnsw/agent/uve/interface_uve_stats_table.cc line = 135 2025-12-11 Thu 01:44:50:764.675 UTC cn-jenkins-deploy-platform-k8s-manifests-278-1 [Thread 139889531872768, Pid 560]: SANDESH: Send FAILED: 1765417490764513 [SYS_NOTICE]: UveVrouterAgent: data= [ name = cn-jenkins-deploy-platform-k8s-manifests-278-1.cluster.local connected_networks= [ [ (*_iter142) = default-domain:default-project:ip-fabric, ] ] vn_count = 1 ] 2025-12-11 Thu 01:44:51:009.748 UTC cn-jenkins-deploy-platform-k8s-manifests-278-1 [Thread 139889531872768, Pid 560]: SANDESH: Send FAILED: 1765417491009571 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-k8s-manifests-278-1.cluster.local:Compute:contrail-vrouter-agent:0 sm_queue_count = 0 sm_stats= [ ev_stats= [ [ [ event = scm::EvCollectorUpdate enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvIdleHoldTimerExpired enqueues = 6 dequeues = 6 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvSandeshSend enqueues = 128 dequeues = 128 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvStart enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpConnectFail enqueues = 7 dequeues = 7 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpDeleteSession enqueues = 7 dequeues = 7 enqueue_fails = 0 dequeue_fails = 0 ], [ event = enqueues = 150 dequeues = 150 enqueue_fails = 0 dequeue_fails = 0 ], ] ] state = Idle state_since = 1765417462021654 last_state = Idle last_event = scm::EvSandeshSend last_event_at = 1765417490764669 ] max_sm_queue_count = 5 ] 2025-12-11 Thu 01:44:51:009.930 UTC cn-jenkins-deploy-platform-k8s-manifests-278-1 [Thread 139889531872768, Pid 560]: SANDESH: Send FAILED: 1765417491009674 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-k8s-manifests-278-1.cluster.local:Compute:contrail-vrouter-agent:0 client_info= [ status = Idle successful_connections = 0 pid = 560 http_port = 8085 start_time = 1765417401006781 collector_name = collector_ip = 10.0.0.22:8086 collector_list= [ [ (*_iter40) = 10.0.0.22:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ComputeCpuStateTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = EndpointSecurityStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 25 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 20 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 61 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UVEVMIStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UVEVMITags [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVMInterfaceAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 3 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 4 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VnObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterControlStatsTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 3 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 62 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 17 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 81, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 128, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-12-11 Thu 01:44:51:014.323 UTC cn-jenkins-deploy-platform-k8s-manifests-278-1 [Thread 139889506682368, Pid 560]: SANDESH: Send FAILED: 1765417491014247 [SYS_NOTICE]: UveVrouterAgent: data= [ name = cn-jenkins-deploy-platform-k8s-manifests-278-1.cluster.local xmpp_peer_list= [ [ [ ip = 10.0.0.22 status = 1 setup_time = 1765417488452504 primary = 1 ], ] ] ] 2025-12-11 Thu 01:44:51:014.470 UTC cn-jenkins-deploy-platform-k8s-manifests-278-1 [Thread 139889506682368, Pid 560]: SANDESH: Send FAILED: 1765417491014287 [SYS_NOTICE]: VrouterStats: data= [ name = cn-jenkins-deploy-platform-k8s-manifests-278-1.cluster.local in_tpkts = 45170 in_bytes = 3177966 out_tpkts = 158641 out_bytes = 1492823761 exception_packets = 43 exception_packets_dropped = 10 exception_packets_allowed = 33 total_flows = 2 raw_vhost_stats= [ name = vhost0 in_pkts = 45170 in_bytes = 3177966 out_pkts = 158641 out_bytes = 1492823761 speed = -1 duplexity = -1 ] raw_vhost_drop_stats= [ ds_discard = 0 ds_pull = 0 ds_invalid_if = 0 ds_invalid_arp = 0 ds_trap_no_if = 0 ds_nowhere_to_go = 0 ds_flow_queue_limit_exceeded = 0 ds_flow_no_memory = 0 ds_flow_invalid_protocol = 0 ds_flow_nat_no_rflow = 0 ds_flow_action_drop = 0 ds_flow_action_invalid = 0 ds_flow_unusable = 0 ds_flow_table_full = 0 ds_interface_tx_discard = 0 ds_interface_drop = 32 ds_duplicated = 0 ds_push = 0 ds_ttl_exceeded = 0 ds_invalid_nh = 0 ds_invalid_label = 0 ds_invalid_protocol = 0 ds_interface_rx_discard = 0 ds_invalid_mcast_source = 0 ds_head_alloc_fail = 0 ds_pcow_fail = 0 ds_mcast_clone_fail = 0 ds_rewrite_fail = 0 ds_misc = 0 ds_invalid_packet = 0 ds_cksum_err = 0 ds_no_fmd = 0 ds_invalid_vnid = 0 ds_frag_err = 0 ds_invalid_source = 0 ds_mcast_df_bit = 0 ds_l2_no_route = 0 ds_vlan_fwd_tx = 0 ds_vlan_fwd_enq = 0 ds_drop_new_flow = 0 ds_flow_evict = 0 ds_trap_original = 0 ds_no_memory = 0 ds_fragment_queue_fail = 0 ds_no_frag_entry = 0 ds_icmp_error = 0 ds_clone_fail = 0 ds_drop_pkts = 32 ] tcp_sport_bitmap= [ [ (*_iter258) = 1, (*_iter258) = 0, (*_iter258) = 0, (*_iter258) = 0, (*_iter258) = 0, (*_iter258) = 2048, (*_iter258) = 0, (*_iter258) = 0, ] ] tcp_dport_bitmap= [ [ (*_iter259) = 1, (*_iter259) = 0, (*_iter259) = 0, (*_iter259) = 0, (*_iter259) = 0, (*_iter259) = 2048, (*_iter259) = 0, (*_iter259) = 0, ] ] raw_drop_stats= [ ds_discard = 0 ds_pull = 0 ds_invalid_if = 0 ds_invalid_arp = 0 ds_trap_no_if = 0 ds_nowhere_to_go = 0 ds_flow_queue_limit_exceeded = 0 ds_flow_no_memory = 0 ds_flow_invalid_protocol = 0 ds_flow_nat_no_rflow = 0 ds_flow_action_drop = 0 ds_flow_action_invalid = 0 ds_flow_unusable = 0 ds_flow_table_full = 0 ds_interface_tx_discard = 0 ds_interface_drop = 32 ds_duplicated = 0 ds_push = 0 ds_ttl_exceeded = 0 ds_invalid_nh = 0 ds_invalid_label = 0 ds_invalid_protocol = 0 ds_interface_rx_discard = 0 ds_invalid_mcast_source = 0 ds_head_alloc_fail = 0 ds_pcow_fail = 0 ds_mcast_clone_fail = 0 ds_rewrite_fail = 0 ds_misc = 0 ds_invalid_packet = 0 ds_cksum_err = 0 ds_no_fmd = 0 ds_invalid_vnid = 0 ds_frag_err = 0 ds_invalid_source = 0 ds_mcast_df_bit = 0 ds_l2_no_route = 0 ds_vlan_fwd_tx = 0 ds_vlan_fwd_enq = 0 ds_drop_new_flow = 0 ds_flow_evict = 0 ds_trap_original = 0 ds_no_memory = 0 ds_fragment_queue_fail = 0 ds_no_frag_entry = 0 ds_icmp_error = 0 ds_clone_fail = 0 ds_drop_pkts = 32 ] total_in_bandwidth_utilization = 4.42869e-10 total_out_bandwidth_utilization = 9.78807e-13 flow_rate= [ added_flows = 0 max_flow_adds_per_second = 0 min_flow_adds_per_second = 0 deleted_flows = 0 max_flow_deletes_per_second = 0 min_flow_deletes_per_second = 0 active_flows = 2 hold_flows = 0 ] phy_band_in_bps= [ [ _iter260->first = eth0 _iter260->second = 81694864, ] ] phy_band_out_bps= [ [ _iter261->first = eth0 _iter261->second = 180558, ] ] raw_phy_if_stats= [ [ _iter262->first = eth0 [ in_pkts = 158560 in_bytes = 1492820359 out_pkts = 45173 out_bytes = 3178092 ], ] ] raw_phy_if_drop_stats= [ [ _iter263->first = eth0 [ ds_discard = 0 ds_pull = 0 ds_invalid_if = 0 ds_invalid_arp = 0 ds_trap_no_if = 0 ds_nowhere_to_go = 0 ds_flow_queue_limit_exceeded = 0 ds_flow_no_memory = 0 ds_flow_invalid_protocol = 0 ds_flow_nat_no_rflow = 0 ds_flow_action_drop = 0 ds_flow_action_invalid = 0 ds_flow_unusable = 0 ds_flow_table_full = 0 ds_interface_tx_discard = 0 ds_interface_drop = 32 ds_duplicated = 0 ds_push = 0 ds_ttl_exceeded = 0 ds_invalid_nh = 0 ds_invalid_label = 0 ds_invalid_protocol = 0 ds_interface_rx_discard = 0 ds_invalid_mcast_source = 0 ds_head_alloc_fail = 0 ds_pcow_fail = 0 ds_mcast_clone_fail = 0 ds_rewrite_fail = 0 ds_misc = 0 ds_invalid_packet = 0 ds_cksum_err = 0 ds_no_fmd = 0 ds_invalid_vnid = 0 ds_frag_err = 0 ds_invalid_source = 0 ds_mcast_df_bit = 0 ds_l2_no_route = 0 ds_vlan_fwd_tx = 0 ds_vlan_fwd_enq = 0 ds_drop_new_flow = 0 ds_flow_evict = 0 ds_trap_original = 0 ds_no_memory = 0 ds_fragment_queue_fail = 0 ds_no_frag_entry = 0 ds_icmp_error = 0 ds_clone_fail = 0 ds_drop_pkts = 32 ], ] ] ] 2025-12-11 Thu 01:44:51:014.528 UTC cn-jenkins-deploy-platform-k8s-manifests-278-1 [Thread 139889506682368, Pid 560]: SANDESH: Send FAILED: 1765417491014496 [SYS_NOTICE]: VrouterControlStatsTrace: data= [ name = cn-jenkins-deploy-platform-k8s-manifests-278-1.cluster.local raw_xmpp_stats= [ [ _iter306->first = 10.0.0.22 [ reconnects = 2 in_msgs = 5 out_msgs = 38 ], ] ] raw_ifmap_stats= [ [ _iter307->first = link_delete_parse_errors _iter307->second = 0, _iter307->first = link_update_parse_errors _iter307->second = 0, _iter307->first = node_delete_parse_errors _iter307->second = 0, _iter307->first = node_update_parse_errors _iter307->second = 0, ] ] raw_rt_table_size= [ [ _iter308->first = default-domain:default-project:ip-fabric:__default__ [ inet4_unicast = 7 inet4_multicast = 0 evpn = 0 bridge = 5 inet6_unicast = 2 ], _iter308->first = default-domain:default-project:ip-fabric:ip-fabric [ inet4_unicast = 2 inet4_multicast = 0 evpn = 2 bridge = 3 inet6_unicast = 3 ], ] ] ] 2025-12-11 Thu 01:44:52:024.875 UTC cn-jenkins-deploy-platform-k8s-manifests-278-1 [Thread 139889519277568, Pid 560]: SANDESH: Send FAILED: 1765417492021854 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-k8s-manifests-278-1.cluster.local process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Functional connection_infos= [ [ [ type = XMPP name = control-node:10.0.0.22 server_addrs= [ [ (*_iter6) = 10.0.0.22:5269, ] ] status = Up description = OpenSent ], [ type = XMPP name = dns-server:10.0.0.22 server_addrs= [ [ (*_iter6) = 10.0.0.22:53, ] ] status = Up description = OpenSent ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.22:8086, ] ] status = Initializing description = Idle : EvIdleHoldTimerExpired -> Connect ], ] ] flag_infos= [ [ ] ] description = Collector connection down ], ] ] ] 2025-12-11 Thu 01:44:52:025.310 UTC cn-jenkins-deploy-platform-k8s-manifests-278-1 [Thread 139889519277568, Pid 560]: SANDESH: Send FAILED: 1765417492024839 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-k8s-manifests-278-1.cluster.local:Compute:contrail-vrouter-agent:0 client_info= [ status = Connect successful_connections = 0 pid = 560 http_port = 8085 start_time = 1765417401006781 collector_name = collector_ip = 10.0.0.22:8086 collector_list= [ [ (*_iter40) = 10.0.0.22:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ComputeCpuStateTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = EndpointSecurityStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 25 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 22 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 62 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UVEVMIStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UVEVMITags [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVMInterfaceAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 3 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 5 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VnObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterControlStatsTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 3 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 4 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 62 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 17 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 82, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 133, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-12-11 Thu 01:44:52:025.354 UTC cn-jenkins-deploy-platform-k8s-manifests-278-1 [Thread 139889519277568, Pid 560]: SANDESH: Send FAILED: 1765417492024920 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-k8s-manifests-278-1.cluster.local process_status= [ [ [ module_id = contrail-vrouter-agent instance_id = 0 state = Functional connection_infos= [ [ [ type = XMPP name = control-node:10.0.0.22 server_addrs= [ [ (*_iter6) = 10.0.0.22:5269, ] ] status = Up description = OpenSent ], [ type = XMPP name = dns-server:10.0.0.22 server_addrs= [ [ (*_iter6) = 10.0.0.22:53, ] ] status = Up description = OpenSent ], [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.22:8086, ] ] status = Initializing description = Connect : EvTcpConnected ], ] ] flag_infos= [ [ ] ] description = Collector connection down ], ] ] ] 2025-12-11 Thu 01:44:52:025.493 UTC cn-jenkins-deploy-platform-k8s-manifests-278-1 [Thread 139889519277568, Pid 560]: SANDESH: Send FAILED: 1765417492024976 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-k8s-manifests-278-1.cluster.local:Compute:contrail-vrouter-agent:0 client_info= [ status = ClientInit successful_connections = 1 pid = 560 http_port = 8085 start_time = 1765417401006781 collector_name = collector_ip = 10.0.0.22:8086 collector_list= [ [ (*_iter40) = 10.0.0.22:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ComputeCpuStateTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = EndpointSecurityStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 26 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 22 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 62 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UVEVMIStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UVEVMITags [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVMInterfaceAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVirtualNetworkAgentTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 3 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveVrouterAgent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 5 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VnObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrfObjectLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterControlStatsTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 3 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = VrouterStats [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 4 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 62 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 17 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 82, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 134, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-12-11 Thu 01:48:00:324.532 UTC cn-jenkins-deploy-platform-k8s-manifests-278-1 [Thread 139889536071168, Pid 560]: Error <19: No such device> querying mac-address for interface Agent-index <8> 2025-12-11 Thu 01:48:07:209.306 UTC cn-jenkins-deploy-platform-k8s-manifests-278-1 [Thread 139889519277568, Pid 560]: ID-PERM not set for object Table <__ifmap__.service_instance.0>. Ignoring it 2025-12-11 Thu 01:48:08:116.513 UTC cn-jenkins-deploy-platform-k8s-manifests-278-1 [Thread 139889527674368, Pid 560]: Error <19: No such device> querying mac-address for interface Agent-index <6> 2025-12-11 Thu 01:48:13:054.124 UTC cn-jenkins-deploy-platform-k8s-manifests-278-1 [Thread 139889536071168, Pid 560]: Error <19: No such device> querying mac-address for interface Agent-index <5> 2025-12-11 Thu 01:52:01:796.616 UTC cn-jenkins-deploy-platform-k8s-manifests-278-1 [Thread 139889510880768, Pid 560]: Error <19: No such device> querying mac-address for interface Agent-index <8> 2025-12-11 Thu 01:52:01:812.039 UTC cn-jenkins-deploy-platform-k8s-manifests-278-1 [Thread 139889506682368, Pid 560]: Error <19: No such device> querying mac-address for interface Agent-index <8> 2025-12-11 Thu 01:52:01:822.447 UTC cn-jenkins-deploy-platform-k8s-manifests-278-1 [Thread 139889506682368, Pid 560]: Error <19: No such device> querying mac-address for interface Agent-index <5> 2025-12-11 Thu 01:52:01:832.221 UTC cn-jenkins-deploy-platform-k8s-manifests-278-1 [Thread 139889506682368, Pid 560]: Error <19: No such device> querying mac-address for interface Agent-index <5> 2025-12-11 Thu 01:52:01:835.552 UTC cn-jenkins-deploy-platform-k8s-manifests-278-1 [Thread 139889506682368, Pid 560]: Error <19: No such device> querying mac-address for interface Agent-index <5> 2025-12-11 Thu 01:52:01:840.214 UTC cn-jenkins-deploy-platform-k8s-manifests-278-1 [Thread 139889506682368, Pid 560]: Error <19: No such device> querying mac-address for interface Agent-index <7> 2025-12-11 Thu 01:52:01:843.353 UTC cn-jenkins-deploy-platform-k8s-manifests-278-1 [Thread 139889506682368, Pid 560]: Error <19: No such device> querying mac-address for interface Agent-index <7> 2025-12-11 Thu 01:52:01:852.698 UTC cn-jenkins-deploy-platform-k8s-manifests-278-1 [Thread 139889536071168, Pid 560]: Error <19: No such device> querying mac-address for interface Agent-index <7> 2025-12-11 Thu 01:52:01:856.064 UTC cn-jenkins-deploy-platform-k8s-manifests-278-1 [Thread 139889536071168, Pid 560]: Error <19: No such device> querying mac-address for interface Agent-index <6> 2025-12-11 Thu 01:52:01:859.033 UTC cn-jenkins-deploy-platform-k8s-manifests-278-1 [Thread 139889536071168, Pid 560]: Error <19: No such device> querying mac-address for interface Agent-index <6> 2025-12-11 Thu 01:52:01:862.608 UTC cn-jenkins-deploy-platform-k8s-manifests-278-1 [Thread 139889536071168, Pid 560]: Error <19: No such device> querying mac-address for interface Agent-index <6> 2025-12-11 Thu 01:52:31:723.672 UTC cn-jenkins-deploy-platform-k8s-manifests-278-1 [Thread 139889519277568, Pid 560]: Error <19: No such device> querying mac-address for interface Agent-index <11> 2025-12-11 Thu 01:52:31:733.475 UTC cn-jenkins-deploy-platform-k8s-manifests-278-1 [Thread 139889506682368, Pid 560]: Error <19: No such device> querying mac-address for interface Agent-index <12> 2025-12-11 Thu 01:52:31:776.845 UTC cn-jenkins-deploy-platform-k8s-manifests-278-1 [Thread 139889531872768, Pid 560]: Error <19: No such device> querying mac-address for interface Agent-index <9>