2024-10-17 Thu 02:10:22:440.315 UTC  cn-jenkins-deploy-platform-k8s-manifests-101-1 [Thread 140693206865664, Pid 809]: KsyncTxQueue CPU pinning policy <>. KsyncTxQueuen not pinned to CPU
2024-10-17 Thu 02:10:22:563.941 UTC  cn-jenkins-deploy-platform-k8s-manifests-101-1 [Thread 140693232056064, Pid 809]: SANDESH: No Client: 1729131022563861  [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-k8s-manifests-101-1.cluster.local:Compute:contrail-vrouter-agent:0 client_info= [  status = Idle successful_connections = 0 pid = 809 http_port = 8085 start_time = 1729131022563768 collector_name =  collector_ip = 0.0.0.0:0 collector_list= [  [ ] ] rx_socket_stats= [  bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration =  errors = 0 ] tx_socket_stats= [  bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration =  errors = 0 ] ] msg_type_agg= [  [ _iter58->first = TcpServerMessageLog [  messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ],  ] ] tx_msg_agg= [  [ _iter59->first = dropped_client_send_failed _iter59->second = 0,  _iter59->first = dropped_header_write_failed _iter59->second = 0,  _iter59->first = dropped_no_client _iter59->second = 0,  _iter59->first = dropped_no_queue _iter59->second = 0,  _iter59->first = dropped_no_session _iter59->second = 0,  _iter59->first = dropped_queue_level _iter59->second = 0,  _iter59->first = dropped_rate_limited _iter59->second = 0,  _iter59->first = dropped_sending_disabled _iter59->second = 1,  _iter59->first = dropped_sending_to_syslog _iter59->second = 0,  _iter59->first = dropped_session_not_connected _iter59->second = 0,  _iter59->first = dropped_validation_failed _iter59->second = 0,  _iter59->first = dropped_write_failed _iter59->second = 0,  _iter59->first = dropped_wrong_client_sm_state _iter59->second = 0,  _iter59->first = sent _iter59->second = 0,  ] ] ]
2024-10-17 Thu 02:10:22:565.187 UTC  cn-jenkins-deploy-platform-k8s-manifests-101-1 [Thread 140693202667264, Pid 809]: SANDESH: Send FAILED: 1729131022564524 AgentVrf [SYS_INFO]: VrfObjectLog: vrf= [ event = Addition  name = default-domain:default-project:ip-fabric:__default__ index = 0 ] file = controller/src/vnsw/agent/oper/vrf.cc line = 414
2024-10-17 Thu 02:10:22:565.224 UTC  cn-jenkins-deploy-platform-k8s-manifests-101-1 [Thread 140693202667264, Pid 809]: SANDESH: Send FAILED: 1729131022564549 AgentVrf [SYS_INFO]: VrfObjectLog: vrf= [ event = Addition  name = default-domain:default-project:ip-fabric:ip-fabric index = 1 ] file = controller/src/vnsw/agent/oper/vrf.cc line = 414
2024-10-17 Thu 02:10:22:565.262 UTC  cn-jenkins-deploy-platform-k8s-manifests-101-1 [Thread 140693202667264, Pid 809]: SANDESH: Send FAILED: 1729131022564632  [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-k8s-manifests-101-1.cluster.local process_status= [  [ [  module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [  [ [  type = Collector name =  server_addrs= [  [ (*_iter6) = 0.0.0.0:0,  ] ] status = Down description = Idle : EvStart -> Disconnect ],  ] ] flag_infos= [  [ ] ] description = Collector connection down, No Configuration for self Number of connections:1, Expected: 3 ],  ] ] ]
2024-10-17 Thu 02:10:22:565.364 UTC  cn-jenkins-deploy-platform-k8s-manifests-101-1 [Thread 140693202667264, Pid 809]: SANDESH: Send FAILED: 1729131022564713  [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-k8s-manifests-101-1.cluster.local:Compute:contrail-vrouter-agent:0 client_info= [  status = Disconnect successful_connections = 0 pid = 809 http_port = 8085 start_time = 1729131022563768 collector_name =  collector_ip = 0.0.0.0:0 collector_list= [  [ ] ] rx_socket_stats= [  bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration =  errors = 0 ] tx_socket_stats= [  bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration =  errors = 0 ] ] msg_type_agg= [  [ _iter58->first = SandeshModuleClientTrace [  messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ],  _iter58->first = TcpServerMessageLog [  messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ],  ] ] tx_msg_agg= [  [ _iter59->first = dropped_client_send_failed _iter59->second = 0,  _iter59->first = dropped_header_write_failed _iter59->second = 0,  _iter59->first = dropped_no_client _iter59->second = 1,  _iter59->first = dropped_no_queue _iter59->second = 0,  _iter59->first = dropped_no_session _iter59->second = 0,  _iter59->first = dropped_queue_level _iter59->second = 0,  _iter59->first = dropped_rate_limited _iter59->second = 0,  _iter59->first = dropped_sending_disabled _iter59->second = 1,  _iter59->first = dropped_sending_to_syslog _iter59->second = 0,  _iter59->first = dropped_session_not_connected _iter59->second = 0,  _iter59->first = dropped_validation_failed _iter59->second = 0,  _iter59->first = dropped_write_failed _iter59->second = 0,  _iter59->first = dropped_wrong_client_sm_state _iter59->second = 0,  _iter59->first = sent _iter59->second = 0,  ] ] ]
2024-10-17 Thu 02:10:22:565.471 UTC  cn-jenkins-deploy-platform-k8s-manifests-101-1 [Thread 140693202667264, Pid 809]: SANDESH: Send FAILED: 1729131022564749  [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-k8s-manifests-101-1.cluster.local:Compute:contrail-vrouter-agent:0 client_info= [  status = Disconnect successful_connections = 0 pid = 809 http_port = 8085 start_time = 1729131022563768 collector_name =  collector_ip = 10.0.0.33:8086 collector_list= [  [ (*_iter40) = 10.0.0.33:8086,  ] ] rx_socket_stats= [  bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration =  errors = 0 ] tx_socket_stats= [  bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration =  errors = 0 ] ] msg_type_agg= [  [ _iter58->first = SandeshModuleClientTrace [  messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ],  _iter58->first = TcpServerMessageLog [  messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ],  ] ] tx_msg_agg= [  [ _iter59->first = dropped_client_send_failed _iter59->second = 0,  _iter59->first = dropped_header_write_failed _iter59->second = 0,  _iter59->first = dropped_no_client _iter59->second = 1,  _iter59->first = dropped_no_queue _iter59->second = 0,  _iter59->first = dropped_no_session _iter59->second = 0,  _iter59->first = dropped_queue_level _iter59->second = 0,  _iter59->first = dropped_rate_limited _iter59->second = 0,  _iter59->first = dropped_sending_disabled _iter59->second = 1,  _iter59->first = dropped_sending_to_syslog _iter59->second = 0,  _iter59->first = dropped_session_not_connected _iter59->second = 0,  _iter59->first = dropped_validation_failed _iter59->second = 0,  _iter59->first = dropped_write_failed _iter59->second = 0,  _iter59->first = dropped_wrong_client_sm_state _iter59->second = 0,  _iter59->first = sent _iter59->second = 0,  ] ] ]
2024-10-17 Thu 02:10:22:565.497 UTC  cn-jenkins-deploy-platform-k8s-manifests-101-1 [Thread 140693202667264, Pid 809]: SANDESH: Send FAILED: 1729131022564771  [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-k8s-manifests-101-1.cluster.local process_status= [  [ [  module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [  [ [  type = Collector name =  server_addrs= [  [ (*_iter6) = 10.0.0.33:8086,  ] ] status = Initializing description = Disconnect : EvCollectorUpdate -> Connect ],  ] ] flag_infos= [  [ ] ] description = Collector connection down, No Configuration for self Number of connections:1, Expected: 3 ],  ] ] ]
2024-10-17 Thu 02:10:22:565.554 UTC  cn-jenkins-deploy-platform-k8s-manifests-101-1 [Thread 140693202667264, Pid 809]: SANDESH: Send FAILED: 1729131022565157  [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-k8s-manifests-101-1.cluster.local:Compute:contrail-vrouter-agent:0 client_info= [  status = Connect successful_connections = 0 pid = 809 http_port = 8085 start_time = 1729131022563768 collector_name =  collector_ip = 10.0.0.33:8086 collector_list= [  [ (*_iter40) = 10.0.0.33:8086,  ] ] rx_socket_stats= [  bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration =  errors = 0 ] tx_socket_stats= [  bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration =  errors = 0 ] ] msg_type_agg= [  [ _iter58->first = SandeshModuleClientTrace [  messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ],  _iter58->first = TcpServerMessageLog [  messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ],  ] ] tx_msg_agg= [  [ _iter59->first = dropped_client_send_failed _iter59->second = 0,  _iter59->first = dropped_header_write_failed _iter59->second = 0,  _iter59->first = dropped_no_client _iter59->second = 1,  _iter59->first = dropped_no_queue _iter59->second = 0,  _iter59->first = dropped_no_session _iter59->second = 0,  _iter59->first = dropped_queue_level _iter59->second = 0,  _iter59->first = dropped_rate_limited _iter59->second = 0,  _iter59->first = dropped_sending_disabled _iter59->second = 1,  _iter59->first = dropped_sending_to_syslog _iter59->second = 0,  _iter59->first = dropped_session_not_connected _iter59->second = 0,  _iter59->first = dropped_validation_failed _iter59->second = 0,  _iter59->first = dropped_write_failed _iter59->second = 0,  _iter59->first = dropped_wrong_client_sm_state _iter59->second = 0,  _iter59->first = sent _iter59->second = 0,  ] ] ]
2024-10-17 Thu 02:10:22:578.037 UTC  cn-jenkins-deploy-platform-k8s-manifests-101-1 [Thread 140693219460864, Pid 809]: SANDESH: Send FAILED: 1729131022577783  [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.0.0.33 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709
2024-10-17 Thu 02:10:22:578.096 UTC  cn-jenkins-deploy-platform-k8s-manifests-101-1 [Thread 140693219460864, Pid 809]: SANDESH: Send FAILED: 1729131022578004  [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-k8s-manifests-101-1.cluster.local process_status= [  [ [  module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [  [ [  type = XMPP name = control-node:10.0.0.33 server_addrs= [  [ (*_iter6) = 10.0.0.33:5269,  ] ] status = Down description = Idle ],  [  type = Collector name =  server_addrs= [  [ (*_iter6) = 10.0.0.33:8086,  ] ] status = Initializing description = Disconnect : EvCollectorUpdate -> Connect ],  ] ] flag_infos= [  [ ] ] description = XMPP:control-node:10.0.0.33, Collector connection down, No Configuration for self Number of connections:2, Expected: 3 ],  ] ] ]
2024-10-17 Thu 02:10:22:578.111 UTC  cn-jenkins-deploy-platform-k8s-manifests-101-1 [Thread 140693219460864, Pid 809]: SANDESH: Send FAILED: 1729131022578089  [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.0.0.33 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709
2024-10-17 Thu 02:10:22:578.136 UTC  cn-jenkins-deploy-platform-k8s-manifests-101-1 [Thread 140693219460864, Pid 809]: SANDESH: Send FAILED: 1729131022578106  [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-k8s-manifests-101-1.cluster.local process_status= [  [ [  module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [  [ [  type = XMPP name = control-node:10.0.0.33 server_addrs= [  [ (*_iter6) = 10.0.0.33:5269,  ] ] status = Down description = Idle ],  [  type = XMPP name = dns-server:10.0.0.33 server_addrs= [  [ (*_iter6) = 10.0.0.33:53,  ] ] status = Down description = Idle ],  [  type = Collector name =  server_addrs= [  [ (*_iter6) = 10.0.0.33:8086,  ] ] status = Initializing description = Disconnect : EvCollectorUpdate -> Connect ],  ] ] flag_infos= [  [ ] ] description = XMPP:control-node:10.0.0.33, XMPP:dns-server:10.0.0.33, Collector connection down, No Configuration for self ],  ] ] ]
2024-10-17 Thu 02:10:24:340.906 UTC  cn-jenkins-deploy-platform-k8s-manifests-101-1 [Thread 140693227857664, Pid 809]: TCP [SYS_ERR]: TcpSessionMessageLog: Session -(-1) > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 314
2024-10-17 Thu 02:10:24:341.045 UTC  cn-jenkins-deploy-platform-k8s-manifests-101-1 [Thread 140693227857664, Pid 809]: SANDESH: Send FAILED: 1729131024340871  [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-k8s-manifests-101-1.cluster.local process_status= [  [ [  module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [  [ [  type = XMPP name = control-node:10.0.0.33 server_addrs= [  [ (*_iter6) = 10.0.0.33:5269,  ] ] status = Down description = Idle ],  [  type = XMPP name = dns-server:10.0.0.33 server_addrs= [  [ (*_iter6) = 10.0.0.33:53,  ] ] status = Down description = Idle ],  [  type = Collector name =  server_addrs= [  [ (*_iter6) = 0.0.0.0:0,  ] ] status = Down description = Connect : EvTcpConnectFail ],  ] ] flag_infos= [  [ ] ] description = XMPP:control-node:10.0.0.33, XMPP:dns-server:10.0.0.33, Collector connection down, No Configuration for self ],  ] ] ]
2024-10-17 Thu 02:10:24:341.195 UTC  cn-jenkins-deploy-platform-k8s-manifests-101-1 [Thread 140693227857664, Pid 809]: SANDESH: Send FAILED: 1729131024341017  [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-k8s-manifests-101-1.cluster.local:Compute:contrail-vrouter-agent:0 client_info= [  status = Idle successful_connections = 0 pid = 809 http_port = 8085 start_time = 1729131022563768 collector_name =  collector_ip = 10.0.0.33:8086 collector_list= [  [ (*_iter40) = 10.0.0.33:8086,  ] ] rx_socket_stats= [  bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration =  errors = 0 ] tx_socket_stats= [  bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration =  errors = 0 ] ] msg_type_agg= [  [ _iter58->first = NodeStatusUVE [  messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 4 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ],  _iter58->first = SandeshModuleClientTrace [  messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 3 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ],  _iter58->first = TcpServerMessageLog [  messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ],  _iter58->first = TcpSessionMessageLog [  messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ],  _iter58->first = VersionInfoLog [  messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ],  _iter58->first = VrfObjectLog [  messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ],  _iter58->first = XmppConnectionLog [  messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ],  ] ] tx_msg_agg= [  [ _iter59->first = dropped_client_send_failed _iter59->second = 0,  _iter59->first = dropped_header_write_failed _iter59->second = 0,  _iter59->first = dropped_no_client _iter59->second = 1,  _iter59->first = dropped_no_queue _iter59->second = 0,  _iter59->first = dropped_no_session _iter59->second = 0,  _iter59->first = dropped_queue_level _iter59->second = 0,  _iter59->first = dropped_rate_limited _iter59->second = 0,  _iter59->first = dropped_sending_disabled _iter59->second = 7,  _iter59->first = dropped_sending_to_syslog _iter59->second = 0,  _iter59->first = dropped_session_not_connected _iter59->second = 0,  _iter59->first = dropped_validation_failed _iter59->second = 0,  _iter59->first = dropped_write_failed _iter59->second = 0,  _iter59->first = dropped_wrong_client_sm_state _iter59->second = 11,  _iter59->first = sent _iter59->second = 0,  ] ] ]
2024-10-17 Thu 02:10:24:341.485 UTC  cn-jenkins-deploy-platform-k8s-manifests-101-1 [Thread 140693376776512, Pid 809]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer  RECV  Mode Client:  Event: Tcp Connect Fail  peer ip: 10.0.0.33 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1302
2024-10-17 Thu 02:10:24:341.514 UTC  cn-jenkins-deploy-platform-k8s-manifests-101-1 [Thread 140693232056064, Pid 809]: SANDESH: Send FAILED: 1729131024341311  [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.0.0.33 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709
2024-10-17 Thu 02:10:24:341.547 UTC  cn-jenkins-deploy-platform-k8s-manifests-101-1 [Thread 140693232056064, Pid 809]: SANDESH: Send FAILED: 1729131024341463  [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.0.0.33 state = Connect event = Connect Event local_port = 33809 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709
2024-10-17 Thu 02:10:24:341.623 UTC  cn-jenkins-deploy-platform-k8s-manifests-101-1 [Thread 140693219460864, Pid 809]: SANDESH: Send FAILED: 1729131024341571  [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.0.0.33 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709
2024-10-17 Thu 02:10:24:341.686 UTC  cn-jenkins-deploy-platform-k8s-manifests-101-1 [Thread 140693219460864, Pid 809]: SANDESH: Send FAILED: 1729131024341674  [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.0.0.33 state = Connect event = Connect Event local_port = 37315 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709
2024-10-17 Thu 02:10:24:341.697 UTC  cn-jenkins-deploy-platform-k8s-manifests-101-1 [Thread 140693232056064, Pid 809]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 314
2024-10-17 Thu 02:10:24:341.724 UTC  cn-jenkins-deploy-platform-k8s-manifests-101-1 [Thread 140693219460864, Pid 809]: SANDESH: Send FAILED: 1729131024341717  [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.0.0.33 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709
2024-10-17 Thu 02:10:24:344.264 UTC  cn-jenkins-deploy-platform-k8s-manifests-101-1 [Thread 140693376776512, Pid 809]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer  RECV  Mode Client:  Event: Tcp Connect Fail  peer ip: 10.0.0.33 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1302
2024-10-17 Thu 02:10:24:344.337 UTC  cn-jenkins-deploy-platform-k8s-manifests-101-1 [Thread 140693211064064, Pid 809]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 314
2024-10-17 Thu 02:10:24:344.382 UTC  cn-jenkins-deploy-platform-k8s-manifests-101-1 [Thread 140693219460864, Pid 809]: SANDESH: Send FAILED: 1729131024344359  [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.0.0.33 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709
2024-10-17 Thu 02:10:25:280.713 UTC  cn-jenkins-deploy-platform-k8s-manifests-101-1 [Thread 140693215262464, Pid 809]: SANDESH: Send FAILED: 1729131025280461  [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.0.0.33 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709
2024-10-17 Thu 02:10:25:280.796 UTC  cn-jenkins-deploy-platform-k8s-manifests-101-1 [Thread 140693215262464, Pid 809]: SANDESH: Send FAILED: 1729131025280692  [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.0.0.33 state = Connect event = Connect Event local_port = 39431 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709
2024-10-17 Thu 02:10:25:280.746 UTC  cn-jenkins-deploy-platform-k8s-manifests-101-1 [Thread 140693376776512, Pid 809]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer  RECV  Mode Client:  Event: Tcp Connect Fail  peer ip: 10.0.0.33 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1302
2024-10-17 Thu 02:10:25:280.869 UTC  cn-jenkins-deploy-platform-k8s-manifests-101-1 [Thread 140693219460864, Pid 809]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 314
2024-10-17 Thu 02:10:25:280.986 UTC  cn-jenkins-deploy-platform-k8s-manifests-101-1 [Thread 140693223659264, Pid 809]: SANDESH: Send FAILED: 1729131025280916  [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.0.0.33 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709
2024-10-17 Thu 02:10:25:331.872 UTC  cn-jenkins-deploy-platform-k8s-manifests-101-1 [Thread 140693202667264, Pid 809]: SANDESH: Send FAILED: 1729131025331826  [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.0.0.33 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709
2024-10-17 Thu 02:10:25:332.040 UTC  cn-jenkins-deploy-platform-k8s-manifests-101-1 [Thread 140693376776512, Pid 809]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer  RECV  Mode Client:  Event: Tcp Connect Fail  peer ip: 10.0.0.33 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1302
2024-10-17 Thu 02:10:25:332.097 UTC  cn-jenkins-deploy-platform-k8s-manifests-101-1 [Thread 140693227857664, Pid 809]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 314
2024-10-17 Thu 02:10:25:332.209 UTC  cn-jenkins-deploy-platform-k8s-manifests-101-1 [Thread 140693202667264, Pid 809]: SANDESH: Send FAILED: 1729131025331979  [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.0.0.33 state = Connect event = Connect Event local_port = 42197 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709
2024-10-17 Thu 02:10:25:332.223 UTC  cn-jenkins-deploy-platform-k8s-manifests-101-1 [Thread 140693202667264, Pid 809]: SANDESH: Send FAILED: 1729131025332140  [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.0.0.33 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709
2024-10-17 Thu 02:10:25:341.671 UTC  cn-jenkins-deploy-platform-k8s-manifests-101-1 [Thread 140693219460864, Pid 809]: SANDESH: Send FAILED: 1729131025341192  [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-k8s-manifests-101-1.cluster.local process_status= [  [ [  module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [  [ [  type = XMPP name = control-node:10.0.0.33 server_addrs= [  [ (*_iter6) = 10.0.0.33:5269,  ] ] status = Down description = Idle ],  [  type = XMPP name = dns-server:10.0.0.33 server_addrs= [  [ (*_iter6) = 10.0.0.33:53,  ] ] status = Down description = Idle ],  [  type = Collector name =  server_addrs= [  [ (*_iter6) = 10.0.0.33:8086,  ] ] status = Initializing description = Idle : EvIdleHoldTimerExpired -> Connect ],  ] ] flag_infos= [  [ ] ] description = XMPP:control-node:10.0.0.33, XMPP:dns-server:10.0.0.33, Collector connection down, No Configuration for self ],  ] ] ]
2024-10-17 Thu 02:10:25:341.775 UTC  cn-jenkins-deploy-platform-k8s-manifests-101-1 [Thread 140693219460864, Pid 809]: TCP [SYS_ERR]: TcpSessionMessageLog: Session -(-1) > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 314
2024-10-17 Thu 02:10:25:342.025 UTC  cn-jenkins-deploy-platform-k8s-manifests-101-1 [Thread 140693219460864, Pid 809]: SANDESH: Send FAILED: 1729131025341629  [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-k8s-manifests-101-1.cluster.local:Compute:contrail-vrouter-agent:0 client_info= [  status = Connect successful_connections = 0 pid = 809 http_port = 8085 start_time = 1729131022563768 collector_name =  collector_ip = 10.0.0.33:8086 collector_list= [  [ (*_iter40) = 10.0.0.33:8086,  ] ] rx_socket_stats= [  bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration =  errors = 0 ] tx_socket_stats= [  bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration =  errors = 0 ] ] msg_type_agg= [  [ _iter58->first = NodeStatusUVE [  messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 5 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ],  _iter58->first = SandeshModuleClientTrace [  messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 4 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ],  _iter58->first = TcpServerMessageLog [  messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ],  _iter58->first = TcpSessionMessageLog [  messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 7 messages_sent_dropped_sending_to_syslog = 0 ],  _iter58->first = VersionInfoLog [  messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ],  _iter58->first = VrfObjectLog [  messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ],  _iter58->first = XmppConnectionLog [  messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 14 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ],  _iter58->first = XmppEventLog [  messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 4 messages_sent_dropped_sending_to_syslog = 0 ],  ] ] tx_msg_agg= [  [ _iter59->first = dropped_client_send_failed _iter59->second = 0,  _iter59->first = dropped_header_write_failed _iter59->second = 0,  _iter59->first = dropped_no_client _iter59->second = 1,  _iter59->first = dropped_no_queue _iter59->second = 0,  _iter59->first = dropped_no_session _iter59->second = 0,  _iter59->first = dropped_queue_level _iter59->second = 0,  _iter59->first = dropped_rate_limited _iter59->second = 0,  _iter59->first = dropped_sending_disabled _iter59->second = 15,  _iter59->first = dropped_sending_to_syslog _iter59->second = 0,  _iter59->first = dropped_session_not_connected _iter59->second = 0,  _iter59->first = dropped_validation_failed _iter59->second = 0,  _iter59->first = dropped_write_failed _iter59->second = 0,  _iter59->first = dropped_wrong_client_sm_state _iter59->second = 25,  _iter59->first = sent _iter59->second = 0,  ] ] ]
2024-10-17 Thu 02:10:25:342.066 UTC  cn-jenkins-deploy-platform-k8s-manifests-101-1 [Thread 140693219460864, Pid 809]: SANDESH: Send FAILED: 1729131025341756  [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-k8s-manifests-101-1.cluster.local process_status= [  [ [  module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [  [ [  type = XMPP name = control-node:10.0.0.33 server_addrs= [  [ (*_iter6) = 10.0.0.33:5269,  ] ] status = Down description = Idle ],  [  type = XMPP name = dns-server:10.0.0.33 server_addrs= [  [ (*_iter6) = 10.0.0.33:53,  ] ] status = Down description = Idle ],  [  type = Collector name =  server_addrs= [  [ (*_iter6) = 0.0.0.0:0,  ] ] status = Down description = Connect : EvTcpConnectFail ],  ] ] flag_infos= [  [ ] ] description = XMPP:control-node:10.0.0.33, XMPP:dns-server:10.0.0.33, Collector connection down, No Configuration for self ],  ] ] ]
2024-10-17 Thu 02:10:25:342.247 UTC  cn-jenkins-deploy-platform-k8s-manifests-101-1 [Thread 140693219460864, Pid 809]: SANDESH: Send FAILED: 1729131025341841  [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-k8s-manifests-101-1.cluster.local:Compute:contrail-vrouter-agent:0 client_info= [  status = Idle successful_connections = 0 pid = 809 http_port = 8085 start_time = 1729131022563768 collector_name =  collector_ip = 10.0.0.33:8086 collector_list= [  [ (*_iter40) = 10.0.0.33:8086,  ] ] rx_socket_stats= [  bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration =  errors = 0 ] tx_socket_stats= [  bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration =  errors = 0 ] ] msg_type_agg= [  [ _iter58->first = NodeStatusUVE [  messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 6 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ],  _iter58->first = SandeshModuleClientTrace [  messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 4 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ],  _iter58->first = TcpServerMessageLog [  messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ],  _iter58->first = TcpSessionMessageLog [  messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 8 messages_sent_dropped_sending_to_syslog = 0 ],  _iter58->first = VersionInfoLog [  messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ],  _iter58->first = VrfObjectLog [  messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ],  _iter58->first = XmppConnectionLog [  messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 14 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ],  _iter58->first = XmppEventLog [  messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 4 messages_sent_dropped_sending_to_syslog = 0 ],  ] ] tx_msg_agg= [  [ _iter59->first = dropped_client_send_failed _iter59->second = 0,  _iter59->first = dropped_header_write_failed _iter59->second = 0,  _iter59->first = dropped_no_client _iter59->second = 1,  _iter59->first = dropped_no_queue _iter59->second = 0,  _iter59->first = dropped_no_session _iter59->second = 0,  _iter59->first = dropped_queue_level _iter59->second = 0,  _iter59->first = dropped_rate_limited _iter59->second = 0,  _iter59->first = dropped_sending_disabled _iter59->second = 16,  _iter59->first = dropped_sending_to_syslog _iter59->second = 0,  _iter59->first = dropped_session_not_connected _iter59->second = 0,  _iter59->first = dropped_validation_failed _iter59->second = 0,  _iter59->first = dropped_write_failed _iter59->second = 0,  _iter59->first = dropped_wrong_client_sm_state _iter59->second = 26,  _iter59->first = sent _iter59->second = 0,  ] ] ]
2024-10-17 Thu 02:10:27:225.092 UTC  cn-jenkins-deploy-platform-k8s-manifests-101-1 [Thread 140693223659264, Pid 809]: SANDESH: Send FAILED: 1729131027225047  [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.0.0.33 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709
2024-10-17 Thu 02:10:27:225.367 UTC  cn-jenkins-deploy-platform-k8s-manifests-101-1 [Thread 140693223659264, Pid 809]: SANDESH: Send FAILED: 1729131027225351  [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.0.0.33 state = Connect event = Connect Event local_port = 42751 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709
2024-10-17 Thu 02:10:27:225.412 UTC  cn-jenkins-deploy-platform-k8s-manifests-101-1 [Thread 140693376776512, Pid 809]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer  RECV  Mode Client:  Event: Tcp Connect Fail  peer ip: 10.0.0.33 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1302
2024-10-17 Thu 02:10:27:225.464 UTC  cn-jenkins-deploy-platform-k8s-manifests-101-1 [Thread 140693223659264, Pid 809]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 314
2024-10-17 Thu 02:10:27:225.504 UTC  cn-jenkins-deploy-platform-k8s-manifests-101-1 [Thread 140693232056064, Pid 809]: SANDESH: Send FAILED: 1729131027225486  [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.0.0.33 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709
2024-10-17 Thu 02:10:27:342.159 UTC  cn-jenkins-deploy-platform-k8s-manifests-101-1 [Thread 140693202667264, Pid 809]: SANDESH: Send FAILED: 1729131027341875  [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-k8s-manifests-101-1.cluster.local process_status= [  [ [  module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [  [ [  type = XMPP name = control-node:10.0.0.33 server_addrs= [  [ (*_iter6) = 10.0.0.33:5269,  ] ] status = Down description = Idle ],  [  type = XMPP name = dns-server:10.0.0.33 server_addrs= [  [ (*_iter6) = 10.0.0.33:53,  ] ] status = Down description = Idle ],  [  type = Collector name =  server_addrs= [  [ (*_iter6) = 10.0.0.33:8086,  ] ] status = Initializing description = Idle : EvIdleHoldTimerExpired -> Connect ],  ] ] flag_infos= [  [ ] ] description = XMPP:control-node:10.0.0.33, XMPP:dns-server:10.0.0.33, Collector connection down, No Configuration for self ],  ] ] ]
2024-10-17 Thu 02:10:27:342.334 UTC  cn-jenkins-deploy-platform-k8s-manifests-101-1 [Thread 140693202667264, Pid 809]: SANDESH: Send FAILED: 1729131027342121  [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-k8s-manifests-101-1.cluster.local:Compute:contrail-vrouter-agent:0 client_info= [  status = Connect successful_connections = 0 pid = 809 http_port = 8085 start_time = 1729131022563768 collector_name =  collector_ip = 10.0.0.33:8086 collector_list= [  [ (*_iter40) = 10.0.0.33:8086,  ] ] rx_socket_stats= [  bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration =  errors = 0 ] tx_socket_stats= [  bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration =  errors = 0 ] ] msg_type_agg= [  [ _iter58->first = NodeStatusUVE [  messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 7 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ],  _iter58->first = SandeshModuleClientTrace [  messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 6 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ],  _iter58->first = TcpServerMessageLog [  messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ],  _iter58->first = TcpSessionMessageLog [  messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 9 messages_sent_dropped_sending_to_syslog = 0 ],  _iter58->first = VersionInfoLog [  messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ],  _iter58->first = VrfObjectLog [  messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ],  _iter58->first = XmppConnectionLog [  messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 17 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ],  _iter58->first = XmppEventLog [  messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 5 messages_sent_dropped_sending_to_syslog = 0 ],  ] ] tx_msg_agg= [  [ _iter59->first = dropped_client_send_failed _iter59->second = 0,  _iter59->first = dropped_header_write_failed _iter59->second = 0,  _iter59->first = dropped_no_client _iter59->second = 1,  _iter59->first = dropped_no_queue _iter59->second = 0,  _iter59->first = dropped_no_session _iter59->second = 0,  _iter59->first = dropped_queue_level _iter59->second = 0,  _iter59->first = dropped_rate_limited _iter59->second = 0,  _iter59->first = dropped_sending_disabled _iter59->second = 18,  _iter59->first = dropped_sending_to_syslog _iter59->second = 0,  _iter59->first = dropped_session_not_connected _iter59->second = 0,  _iter59->first = dropped_validation_failed _iter59->second = 0,  _iter59->first = dropped_write_failed _iter59->second = 0,  _iter59->first = dropped_wrong_client_sm_state _iter59->second = 32,  _iter59->first = sent _iter59->second = 0,  ] ] ]
2024-10-17 Thu 02:10:27:342.389 UTC  cn-jenkins-deploy-platform-k8s-manifests-101-1 [Thread 140693202667264, Pid 809]: TCP [SYS_ERR]: TcpSessionMessageLog: Session -(-1) > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 314
2024-10-17 Thu 02:10:27:342.468 UTC  cn-jenkins-deploy-platform-k8s-manifests-101-1 [Thread 140693202667264, Pid 809]: SANDESH: Send FAILED: 1729131027342371  [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-k8s-manifests-101-1.cluster.local process_status= [  [ [  module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [  [ [  type = XMPP name = control-node:10.0.0.33 server_addrs= [  [ (*_iter6) = 10.0.0.33:5269,  ] ] status = Down description = Idle ],  [  type = XMPP name = dns-server:10.0.0.33 server_addrs= [  [ (*_iter6) = 10.0.0.33:53,  ] ] status = Down description = Idle ],  [  type = Collector name =  server_addrs= [  [ (*_iter6) = 0.0.0.0:0,  ] ] status = Down description = Connect : EvTcpConnectFail ],  ] ] flag_infos= [  [ ] ] description = XMPP:control-node:10.0.0.33, XMPP:dns-server:10.0.0.33, Collector connection down, No Configuration for self ],  ] ] ]
2024-10-17 Thu 02:10:27:342.618 UTC  cn-jenkins-deploy-platform-k8s-manifests-101-1 [Thread 140693202667264, Pid 809]: SANDESH: Send FAILED: 1729131027342450  [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-k8s-manifests-101-1.cluster.local:Compute:contrail-vrouter-agent:0 client_info= [  status = Idle successful_connections = 0 pid = 809 http_port = 8085 start_time = 1729131022563768 collector_name =  collector_ip = 10.0.0.33:8086 collector_list= [  [ (*_iter40) = 10.0.0.33:8086,  ] ] rx_socket_stats= [  bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration =  errors = 0 ] tx_socket_stats= [  bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration =  errors = 0 ] ] msg_type_agg= [  [ _iter58->first = NodeStatusUVE [  messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 8 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ],  _iter58->first = SandeshModuleClientTrace [  messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 7 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ],  _iter58->first = TcpServerMessageLog [  messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ],  _iter58->first = TcpSessionMessageLog [  messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 10 messages_sent_dropped_sending_to_syslog = 0 ],  _iter58->first = VersionInfoLog [  messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ],  _iter58->first = VrfObjectLog [  messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ],  _iter58->first = XmppConnectionLog [  messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 17 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ],  _iter58->first = XmppEventLog [  messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 5 messages_sent_dropped_sending_to_syslog = 0 ],  ] ] tx_msg_agg= [  [ _iter59->first = dropped_client_send_failed _iter59->second = 0,  _iter59->first = dropped_header_write_failed _iter59->second = 0,  _iter59->first = dropped_no_client _iter59->second = 1,  _iter59->first = dropped_no_queue _iter59->second = 0,  _iter59->first = dropped_no_session _iter59->second = 0,  _iter59->first = dropped_queue_level _iter59->second = 0,  _iter59->first = dropped_rate_limited _iter59->second = 0,  _iter59->first = dropped_sending_disabled _iter59->second = 19,  _iter59->first = dropped_sending_to_syslog _iter59->second = 0,  _iter59->first = dropped_session_not_connected _iter59->second = 0,  _iter59->first = dropped_validation_failed _iter59->second = 0,  _iter59->first = dropped_write_failed _iter59->second = 0,  _iter59->first = dropped_wrong_client_sm_state _iter59->second = 34,  _iter59->first = sent _iter59->second = 0,  ] ] ]
2024-10-17 Thu 02:10:27:438.354 UTC  cn-jenkins-deploy-platform-k8s-manifests-101-1 [Thread 140693232056064, Pid 809]: SANDESH: Send FAILED: 1729131027438306  [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.0.0.33 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709
2024-10-17 Thu 02:10:27:438.598 UTC  cn-jenkins-deploy-platform-k8s-manifests-101-1 [Thread 140693232056064, Pid 809]: SANDESH: Send FAILED: 1729131027438553  [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.0.0.33 state = Connect event = Connect Event local_port = 44937 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709
2024-10-17 Thu 02:10:27:438.678 UTC  cn-jenkins-deploy-platform-k8s-manifests-101-1 [Thread 140693376776512, Pid 809]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer  RECV  Mode Client:  Event: Tcp Connect Fail  peer ip: 10.0.0.33 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1302
2024-10-17 Thu 02:10:27:438.757 UTC  cn-jenkins-deploy-platform-k8s-manifests-101-1 [Thread 140693227857664, Pid 809]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 314
2024-10-17 Thu 02:10:27:438.811 UTC  cn-jenkins-deploy-platform-k8s-manifests-101-1 [Thread 140693219460864, Pid 809]: SANDESH: Send FAILED: 1729131027438797  [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.0.0.33 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709
2024-10-17 Thu 02:10:30:825.684 UTC  cn-jenkins-deploy-platform-k8s-manifests-101-1 [Thread 140693211064064, Pid 809]: SANDESH: Send FAILED: 1729131030825630  [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.0.0.33 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709
2024-10-17 Thu 02:10:30:825.858 UTC  cn-jenkins-deploy-platform-k8s-manifests-101-1 [Thread 140693376776512, Pid 809]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer  RECV  Mode Client:  Event: Tcp Connect Fail  peer ip: 10.0.0.33 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1302
2024-10-17 Thu 02:10:30:825.926 UTC  cn-jenkins-deploy-platform-k8s-manifests-101-1 [Thread 140693202667264, Pid 809]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 314
2024-10-17 Thu 02:10:30:825.932 UTC  cn-jenkins-deploy-platform-k8s-manifests-101-1 [Thread 140693215262464, Pid 809]: SANDESH: Send FAILED: 1729131030825895  [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.0.0.33 state = Connect event = Connect Event local_port = 38157 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709
2024-10-17 Thu 02:10:30:825.960 UTC  cn-jenkins-deploy-platform-k8s-manifests-101-1 [Thread 140693215262464, Pid 809]: SANDESH: Send FAILED: 1729131030825951  [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.0.0.33 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709
2024-10-17 Thu 02:10:30:825.973 UTC  cn-jenkins-deploy-platform-k8s-manifests-101-1 [Thread 140693215262464, Pid 809]: SANDESH: Send FAILED: 1729131030825965  [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.0.0.33 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709
2024-10-17 Thu 02:10:30:826.043 UTC  cn-jenkins-deploy-platform-k8s-manifests-101-1 [Thread 140693202667264, Pid 809]: SANDESH: Send FAILED: 1729131030826009  [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-k8s-manifests-101-1.cluster.local process_status= [  [ [  module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [  [ [  type = XMPP name = control-node:10.0.0.33 server_addrs= [  [ (*_iter6) = 10.0.0.33:5269,  ] ] status = Down description = Connect ],  [  type = XMPP name = dns-server:10.0.0.33 server_addrs= [  [ (*_iter6) = 10.0.0.33:53,  ] ] status = Down description = Idle ],  [  type = Collector name =  server_addrs= [  [ (*_iter6) = 0.0.0.0:0,  ] ] status = Down description = Connect : EvTcpConnectFail ],  ] ] flag_infos= [  [ ] ] description = XMPP:control-node:10.0.0.33, XMPP:dns-server:10.0.0.33, Collector connection down, No Configuration for self ],  ] ] ]
2024-10-17 Thu 02:10:31:344.430 UTC  cn-jenkins-deploy-platform-k8s-manifests-101-1 [Thread 140693211064064, Pid 809]: SANDESH: Send FAILED: 1729131031342508  [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-k8s-manifests-101-1.cluster.local process_status= [  [ [  module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [  [ [  type = XMPP name = control-node:10.0.0.33 server_addrs= [  [ (*_iter6) = 10.0.0.33:5269,  ] ] status = Down description = Connect ],  [  type = XMPP name = dns-server:10.0.0.33 server_addrs= [  [ (*_iter6) = 10.0.0.33:53,  ] ] status = Down description = Idle ],  [  type = Collector name =  server_addrs= [  [ (*_iter6) = 10.0.0.33:8086,  ] ] status = Initializing description = Idle : EvIdleHoldTimerExpired -> Connect ],  ] ] flag_infos= [  [ ] ] description = XMPP:control-node:10.0.0.33, XMPP:dns-server:10.0.0.33, Collector connection down, No Configuration for self ],  ] ] ]
2024-10-17 Thu 02:10:31:344.608 UTC  cn-jenkins-deploy-platform-k8s-manifests-101-1 [Thread 140693211064064, Pid 809]: TCP [SYS_ERR]: TcpSessionMessageLog: Session -(-1) > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 314
2024-10-17 Thu 02:10:31:344.876 UTC  cn-jenkins-deploy-platform-k8s-manifests-101-1 [Thread 140693211064064, Pid 809]: SANDESH: Send FAILED: 1729131031344383  [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-k8s-manifests-101-1.cluster.local:Compute:contrail-vrouter-agent:0 client_info= [  status = Connect successful_connections = 0 pid = 809 http_port = 8085 start_time = 1729131022563768 collector_name =  collector_ip = 10.0.0.33:8086 collector_list= [  [ (*_iter40) = 10.0.0.33:8086,  ] ] rx_socket_stats= [  bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration =  errors = 0 ] tx_socket_stats= [  bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration =  errors = 0 ] ] msg_type_agg= [  [ _iter58->first = NodeStatusUVE [  messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 10 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ],  _iter58->first = SandeshModuleClientTrace [  messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 8 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ],  _iter58->first = TcpServerMessageLog [  messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ],  _iter58->first = TcpSessionMessageLog [  messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 14 messages_sent_dropped_sending_to_syslog = 0 ],  _iter58->first = VersionInfoLog [  messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ],  _iter58->first = VrfObjectLog [  messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ],  _iter58->first = XmppConnectionLog [  messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 24 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ],  _iter58->first = XmppEventLog [  messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 7 messages_sent_dropped_sending_to_syslog = 0 ],  ] ] tx_msg_agg= [  [ _iter59->first = dropped_client_send_failed _iter59->second = 0,  _iter59->first = dropped_header_write_failed _iter59->second = 0,  _iter59->first = dropped_no_client _iter59->second = 1,  _iter59->first = dropped_no_queue _iter59->second = 0,  _iter59->first = dropped_no_session _iter59->second = 0,  _iter59->first = dropped_queue_level _iter59->second = 0,  _iter59->first = dropped_rate_limited _iter59->second = 0,  _iter59->first = dropped_sending_disabled _iter59->second = 25,  _iter59->first = dropped_sending_to_syslog _iter59->second = 0,  _iter59->first = dropped_session_not_connected _iter59->second = 0,  _iter59->first = dropped_validation_failed _iter59->second = 0,  _iter59->first = dropped_write_failed _iter59->second = 0,  _iter59->first = dropped_wrong_client_sm_state _iter59->second = 44,  _iter59->first = sent _iter59->second = 0,  ] ] ]
2024-10-17 Thu 02:10:31:344.907 UTC  cn-jenkins-deploy-platform-k8s-manifests-101-1 [Thread 140693211064064, Pid 809]: SANDESH: Send FAILED: 1729131031344549  [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-k8s-manifests-101-1.cluster.local process_status= [  [ [  module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [  [ [  type = XMPP name = control-node:10.0.0.33 server_addrs= [  [ (*_iter6) = 10.0.0.33:5269,  ] ] status = Down description = Connect ],  [  type = XMPP name = dns-server:10.0.0.33 server_addrs= [  [ (*_iter6) = 10.0.0.33:53,  ] ] status = Down description = Idle ],  [  type = Collector name =  server_addrs= [  [ (*_iter6) = 0.0.0.0:0,  ] ] status = Down description = Connect : EvTcpConnectFail ],  ] ] flag_infos= [  [ ] ] description = XMPP:control-node:10.0.0.33, XMPP:dns-server:10.0.0.33, Collector connection down, No Configuration for self ],  ] ] ]
2024-10-17 Thu 02:10:31:345.038 UTC  cn-jenkins-deploy-platform-k8s-manifests-101-1 [Thread 140693211064064, Pid 809]: SANDESH: Send FAILED: 1729131031344748  [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-k8s-manifests-101-1.cluster.local:Compute:contrail-vrouter-agent:0 client_info= [  status = Idle successful_connections = 0 pid = 809 http_port = 8085 start_time = 1729131022563768 collector_name =  collector_ip = 10.0.0.33:8086 collector_list= [  [ (*_iter40) = 10.0.0.33:8086,  ] ] rx_socket_stats= [  bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration =  errors = 0 ] tx_socket_stats= [  bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration =  errors = 0 ] ] msg_type_agg= [  [ _iter58->first = NodeStatusUVE [  messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 11 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ],  _iter58->first = SandeshModuleClientTrace [  messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 8 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ],  _iter58->first = TcpServerMessageLog [  messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ],  _iter58->first = TcpSessionMessageLog [  messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 15 messages_sent_dropped_sending_to_syslog = 0 ],  _iter58->first = VersionInfoLog [  messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ],  _iter58->first = VrfObjectLog [  messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ],  _iter58->first = XmppConnectionLog [  messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 24 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ],  _iter58->first = XmppEventLog [  messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 7 messages_sent_dropped_sending_to_syslog = 0 ],  ] ] tx_msg_agg= [  [ _iter59->first = dropped_client_send_failed _iter59->second = 0,  _iter59->first = dropped_header_write_failed _iter59->second = 0,  _iter59->first = dropped_no_client _iter59->second = 1,  _iter59->first = dropped_no_queue _iter59->second = 0,  _iter59->first = dropped_no_session _iter59->second = 0,  _iter59->first = dropped_queue_level _iter59->second = 0,  _iter59->first = dropped_rate_limited _iter59->second = 0,  _iter59->first = dropped_sending_disabled _iter59->second = 26,  _iter59->first = dropped_sending_to_syslog _iter59->second = 0,  _iter59->first = dropped_session_not_connected _iter59->second = 0,  _iter59->first = dropped_validation_failed _iter59->second = 0,  _iter59->first = dropped_write_failed _iter59->second = 0,  _iter59->first = dropped_wrong_client_sm_state _iter59->second = 45,  _iter59->first = sent _iter59->second = 0,  ] ] ]
2024-10-17 Thu 02:10:31:542.972 UTC  cn-jenkins-deploy-platform-k8s-manifests-101-1 [Thread 140693232056064, Pid 809]: SANDESH: Send FAILED: 1729131031542930  [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.0.0.33 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709
2024-10-17 Thu 02:10:31:543.383 UTC  cn-jenkins-deploy-platform-k8s-manifests-101-1 [Thread 140693376776512, Pid 809]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer  RECV  Mode Client:  Event: Tcp Connect Fail  peer ip: 10.0.0.33 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1302
2024-10-17 Thu 02:10:31:543.462 UTC  cn-jenkins-deploy-platform-k8s-manifests-101-1 [Thread 140693227857664, Pid 809]: SANDESH: Send FAILED: 1729131031543419  [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.0.0.33 state = Connect event = Connect Event local_port = 44557 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709
2024-10-17 Thu 02:10:31:543.520 UTC  cn-jenkins-deploy-platform-k8s-manifests-101-1 [Thread 140693202667264, Pid 809]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 314
2024-10-17 Thu 02:10:31:543.554 UTC  cn-jenkins-deploy-platform-k8s-manifests-101-1 [Thread 140693215262464, Pid 809]: SANDESH: Send FAILED: 1729131031543543  [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.0.0.33 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709
2024-10-17 Thu 02:10:31:543.579 UTC  cn-jenkins-deploy-platform-k8s-manifests-101-1 [Thread 140693215262464, Pid 809]: SANDESH: Send FAILED: 1729131031543547  [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.0.0.33 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709
2024-10-17 Thu 02:10:31:543.644 UTC  cn-jenkins-deploy-platform-k8s-manifests-101-1 [Thread 140693232056064, Pid 809]: SANDESH: Send FAILED: 1729131031543616  [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-k8s-manifests-101-1.cluster.local process_status= [  [ [  module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [  [ [  type = XMPP name = control-node:10.0.0.33 server_addrs= [  [ (*_iter6) = 10.0.0.33:5269,  ] ] status = Down description = Connect ],  [  type = XMPP name = dns-server:10.0.0.33 server_addrs= [  [ (*_iter6) = 10.0.0.33:53,  ] ] status = Down description = Connect ],  [  type = Collector name =  server_addrs= [  [ (*_iter6) = 0.0.0.0:0,  ] ] status = Down description = Connect : EvTcpConnectFail ],  ] ] flag_infos= [  [ ] ] description = XMPP:control-node:10.0.0.33, XMPP:dns-server:10.0.0.33, Collector connection down, No Configuration for self ],  ] ] ]
2024-10-17 Thu 02:10:38:242.173 UTC  cn-jenkins-deploy-platform-k8s-manifests-101-1 [Thread 140693232056064, Pid 809]: SANDESH: Send FAILED: 1729131038242119  [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.0.0.33 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709
2024-10-17 Thu 02:10:38:242.425 UTC  cn-jenkins-deploy-platform-k8s-manifests-101-1 [Thread 140693376776512, Pid 809]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer  RECV  Mode Client:  Event: Tcp Connect Fail  peer ip: 10.0.0.33 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1302
2024-10-17 Thu 02:10:38:242.519 UTC  cn-jenkins-deploy-platform-k8s-manifests-101-1 [Thread 140693215262464, Pid 809]: SANDESH: Send FAILED: 1729131038242478  [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.0.0.33 state = Connect event = Connect Event local_port = 38245 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709
2024-10-17 Thu 02:10:38:242.532 UTC  cn-jenkins-deploy-platform-k8s-manifests-101-1 [Thread 140693227857664, Pid 809]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 314
2024-10-17 Thu 02:10:38:242.590 UTC  cn-jenkins-deploy-platform-k8s-manifests-101-1 [Thread 140693202667264, Pid 809]: SANDESH: Send FAILED: 1729131038242578  [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.0.0.33 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709
2024-10-17 Thu 02:10:38:242.607 UTC  cn-jenkins-deploy-platform-k8s-manifests-101-1 [Thread 140693202667264, Pid 809]: SANDESH: Send FAILED: 1729131038242585  [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.0.0.33 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709
2024-10-17 Thu 02:10:39:345.875 UTC  cn-jenkins-deploy-platform-k8s-manifests-101-1 [Thread 140693215262464, Pid 809]: SANDESH: Send FAILED: 1729131039344788  [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-k8s-manifests-101-1.cluster.local process_status= [  [ [  module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [  [ [  type = XMPP name = control-node:10.0.0.33 server_addrs= [  [ (*_iter6) = 10.0.0.33:5269,  ] ] status = Down description = Connect ],  [  type = XMPP name = dns-server:10.0.0.33 server_addrs= [  [ (*_iter6) = 10.0.0.33:53,  ] ] status = Down description = Connect ],  [  type = Collector name =  server_addrs= [  [ (*_iter6) = 10.0.0.33:8086,  ] ] status = Initializing description = Idle : EvIdleHoldTimerExpired -> Connect ],  ] ] flag_infos= [  [ ] ] description = XMPP:control-node:10.0.0.33, XMPP:dns-server:10.0.0.33, Collector connection down, No Configuration for self ],  ] ] ]
2024-10-17 Thu 02:10:39:345.945 UTC  cn-jenkins-deploy-platform-k8s-manifests-101-1 [Thread 140693215262464, Pid 809]: TCP [SYS_ERR]: TcpSessionMessageLog: Session -(-1) > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 314
2024-10-17 Thu 02:10:39:346.122 UTC  cn-jenkins-deploy-platform-k8s-manifests-101-1 [Thread 140693215262464, Pid 809]: SANDESH: Send FAILED: 1729131039345836  [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-k8s-manifests-101-1.cluster.local:Compute:contrail-vrouter-agent:0 client_info= [  status = Connect successful_connections = 0 pid = 809 http_port = 8085 start_time = 1729131022563768 collector_name =  collector_ip = 10.0.0.33:8086 collector_list= [  [ (*_iter40) = 10.0.0.33:8086,  ] ] rx_socket_stats= [  bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration =  errors = 0 ] tx_socket_stats= [  bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration =  errors = 0 ] ] msg_type_agg= [  [ _iter58->first = NodeStatusUVE [  messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 13 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ],  _iter58->first = SandeshModuleClientTrace [  messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 10 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ],  _iter58->first = TcpServerMessageLog [  messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ],  _iter58->first = TcpSessionMessageLog [  messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 19 messages_sent_dropped_sending_to_syslog = 0 ],  _iter58->first = VersionInfoLog [  messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ],  _iter58->first = VrfObjectLog [  messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ],  _iter58->first = XmppConnectionLog [  messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 32 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ],  _iter58->first = XmppEventLog [  messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 9 messages_sent_dropped_sending_to_syslog = 0 ],  ] ] tx_msg_agg= [  [ _iter59->first = dropped_client_send_failed _iter59->second = 0,  _iter59->first = dropped_header_write_failed _iter59->second = 0,  _iter59->first = dropped_no_client _iter59->second = 1,  _iter59->first = dropped_no_queue _iter59->second = 0,  _iter59->first = dropped_no_session _iter59->second = 0,  _iter59->first = dropped_queue_level _iter59->second = 0,  _iter59->first = dropped_rate_limited _iter59->second = 0,  _iter59->first = dropped_sending_disabled _iter59->second = 32,  _iter59->first = dropped_sending_to_syslog _iter59->second = 0,  _iter59->first = dropped_session_not_connected _iter59->second = 0,  _iter59->first = dropped_validation_failed _iter59->second = 0,  _iter59->first = dropped_write_failed _iter59->second = 0,  _iter59->first = dropped_wrong_client_sm_state _iter59->second = 57,  _iter59->first = sent _iter59->second = 0,  ] ] ]
2024-10-17 Thu 02:10:39:346.153 UTC  cn-jenkins-deploy-platform-k8s-manifests-101-1 [Thread 140693215262464, Pid 809]: SANDESH: Send FAILED: 1729131039345928  [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-k8s-manifests-101-1.cluster.local process_status= [  [ [  module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [  [ [  type = XMPP name = control-node:10.0.0.33 server_addrs= [  [ (*_iter6) = 10.0.0.33:5269,  ] ] status = Down description = Connect ],  [  type = XMPP name = dns-server:10.0.0.33 server_addrs= [  [ (*_iter6) = 10.0.0.33:53,  ] ] status = Down description = Connect ],  [  type = Collector name =  server_addrs= [  [ (*_iter6) = 0.0.0.0:0,  ] ] status = Down description = Connect : EvTcpConnectFail ],  ] ] flag_infos= [  [ ] ] description = XMPP:control-node:10.0.0.33, XMPP:dns-server:10.0.0.33, Collector connection down, No Configuration for self ],  ] ] ]
2024-10-17 Thu 02:10:39:346.283 UTC  cn-jenkins-deploy-platform-k8s-manifests-101-1 [Thread 140693215262464, Pid 809]: SANDESH: Send FAILED: 1729131039346000  [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-k8s-manifests-101-1.cluster.local:Compute:contrail-vrouter-agent:0 client_info= [  status = Idle successful_connections = 0 pid = 809 http_port = 8085 start_time = 1729131022563768 collector_name =  collector_ip = 10.0.0.33:8086 collector_list= [  [ (*_iter40) = 10.0.0.33:8086,  ] ] rx_socket_stats= [  bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration =  errors = 0 ] tx_socket_stats= [  bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration =  errors = 0 ] ] msg_type_agg= [  [ _iter58->first = NodeStatusUVE [  messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 14 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ],  _iter58->first = SandeshModuleClientTrace [  messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 10 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ],  _iter58->first = TcpServerMessageLog [  messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ],  _iter58->first = TcpSessionMessageLog [  messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 20 messages_sent_dropped_sending_to_syslog = 0 ],  _iter58->first = VersionInfoLog [  messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ],  _iter58->first = VrfObjectLog [  messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ],  _iter58->first = XmppConnectionLog [  messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 32 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ],  _iter58->first = XmppEventLog [  messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 9 messages_sent_dropped_sending_to_syslog = 0 ],  ] ] tx_msg_agg= [  [ _iter59->first = dropped_client_send_failed _iter59->second = 0,  _iter59->first = dropped_header_write_failed _iter59->second = 0,  _iter59->first = dropped_no_client _iter59->second = 1,  _iter59->first = dropped_no_queue _iter59->second = 0,  _iter59->first = dropped_no_session _iter59->second = 0,  _iter59->first = dropped_queue_level _iter59->second = 0,  _iter59->first = dropped_rate_limited _iter59->second = 0,  _iter59->first = dropped_sending_disabled _iter59->second = 33,  _iter59->first = dropped_sending_to_syslog _iter59->second = 0,  _iter59->first = dropped_session_not_connected _iter59->second = 0,  _iter59->first = dropped_validation_failed _iter59->second = 0,  _iter59->first = dropped_write_failed _iter59->second = 0,  _iter59->first = dropped_wrong_client_sm_state _iter59->second = 58,  _iter59->first = sent _iter59->second = 0,  ] ] ]
2024-10-17 Thu 02:10:39:751.806 UTC  cn-jenkins-deploy-platform-k8s-manifests-101-1 [Thread 140693215262464, Pid 809]: SANDESH: Send FAILED: 1729131039751663  [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.0.0.33 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709
2024-10-17 Thu 02:10:39:752.178 UTC  cn-jenkins-deploy-platform-k8s-manifests-101-1 [Thread 140693376776512, Pid 809]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer  RECV  Mode Client:  Event: Tcp Connect Fail  peer ip: 10.0.0.33 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1302
2024-10-17 Thu 02:10:39:752.266 UTC  cn-jenkins-deploy-platform-k8s-manifests-101-1 [Thread 140693227857664, Pid 809]: SANDESH: Send FAILED: 1729131039752216  [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.0.0.33 state = Connect event = Connect Event local_port = 36543 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709
2024-10-17 Thu 02:10:39:753.105 UTC  cn-jenkins-deploy-platform-k8s-manifests-101-1 [Thread 140693223659264, Pid 809]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 314
2024-10-17 Thu 02:10:39:753.184 UTC  cn-jenkins-deploy-platform-k8s-manifests-101-1 [Thread 140693232056064, Pid 809]: SANDESH: Send FAILED: 1729131039753134  [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.0.0.33 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709
2024-10-17 Thu 02:10:39:753.197 UTC  cn-jenkins-deploy-platform-k8s-manifests-101-1 [Thread 140693232056064, Pid 809]: SANDESH: Send FAILED: 1729131039753143  [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.0.0.33 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709
2024-10-17 Thu 02:10:52:309.619 UTC  cn-jenkins-deploy-platform-k8s-manifests-101-1 [Thread 140693223659264, Pid 809]: SANDESH: Send FAILED: 1729131052309575  [SYS_NOTICE]: UveVrouterAgent: data= [ name = cn-jenkins-deploy-platform-k8s-manifests-101-1.cluster.local interface_list= [  [ ] ] error_intf_list= [  [ ] ] no_config_intf_list= [  [ (*_iter151) = 00000000-0000-0000-0000-000000000000,  ] ] vmi_count= [  active = 1 ] down_interface_count = 1 ]
2024-10-17 Thu 02:10:52:310.613 UTC  cn-jenkins-deploy-platform-k8s-manifests-101-1 [Thread 140693232056064, Pid 809]: SANDESH: Send FAILED: 1729131052310507  [SYS_NOTICE]: UveVirtualNetworkAgentTrace: data= [ name = __UNKNOWN__ vrf_stats_list= [  [ [  name = __untitled__ diag_packet_count = 0 unknown_unicast_floods = 0 arp_packet_counts= [  from_vm_interface= [ stats= [ proxies = 0 stitches = 0 floods = 0 ] ] from_physical_interface= [  stats= [ proxies = 0 stitches = 0 floods = 0 ] ] ] offload_packet_counts= [  gro = 0 ] nh_packet_counts= [  discards = 0 resolves = 0 l3_receives = 0 l2_receives = 0 local_vm_l3_forwards = 0 local_vm_l2_forwards = 0 vrf_translates = 0 ecmp_forwards = 0 tunnel_nh_stats= [  udp_encaps = 0 mpls_over_udp_encaps = 0 mpls_over_gre_encaps = 0 vxlan_encaps = 0 ] comp_nh_stats= [  source_replication_forwards = 0 edge_replication_forwards = 0 local_vm_l3_forwards = 0 total_multicast_forwards = 0 ] ] ],  ] ] ]
2024-10-17 Thu 02:10:52:564.213 UTC  cn-jenkins-deploy-platform-k8s-manifests-101-1 [Thread 140693219460864, Pid 809]: SANDESH: Send FAILED: 1729131052564060  [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-k8s-manifests-101-1.cluster.local:Compute:contrail-vrouter-agent:0 sm_queue_count = 0 sm_stats= [  ev_stats= [  [ [  event = scm::EvCollectorUpdate enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ],  [  event = scm::EvIdleHoldTimerExpired enqueues = 4 dequeues = 4 enqueue_fails = 0 dequeue_fails = 0 ],  [  event = scm::EvSandeshSend enqueues = 67 dequeues = 67 enqueue_fails = 0 dequeue_fails = 0 ],  [  event = scm::EvStart enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ],  [  event = scm::EvTcpConnectFail enqueues = 5 dequeues = 5 enqueue_fails = 0 dequeue_fails = 0 ],  [  event = scm::EvTcpDeleteSession enqueues = 5 dequeues = 5 enqueue_fails = 0 dequeue_fails = 0 ],  [  event =  enqueues = 83 dequeues = 83 enqueue_fails = 0 dequeue_fails = 0 ],  ] ] state = Idle state_since = 1729131039345970 last_state = Idle last_event = scm::EvSandeshSend last_event_at = 1729131052310536 ] max_sm_queue_count = 7 ]
2024-10-17 Thu 02:10:52:564.388 UTC  cn-jenkins-deploy-platform-k8s-manifests-101-1 [Thread 140693219460864, Pid 809]: SANDESH: Send FAILED: 1729131052564149  [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-k8s-manifests-101-1.cluster.local:Compute:contrail-vrouter-agent:0 client_info= [  status = Idle successful_connections = 0 pid = 809 http_port = 8085 start_time = 1729131022563768 collector_name =  collector_ip = 10.0.0.33:8086 collector_list= [  [ (*_iter40) = 10.0.0.33:8086,  ] ] rx_socket_stats= [  bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration =  errors = 0 ] tx_socket_stats= [  bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration =  errors = 0 ] ] msg_type_agg= [  [ _iter58->first = NodeStatusUVE [  messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 15 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ],  _iter58->first = SandeshModuleClientTrace [  messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 12 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ],  _iter58->first = TcpServerMessageLog [  messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ],  _iter58->first = TcpSessionMessageLog [  messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 27 messages_sent_dropped_sending_to_syslog = 0 ],  _iter58->first = UveVirtualNetworkAgentTrace [  messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ],  _iter58->first = UveVrouterAgent [  messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ],  _iter58->first = VersionInfoLog [  messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ],  _iter58->first = VrfObjectLog [  messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ],  _iter58->first = XmppConnectionLog [  messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 36 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ],  _iter58->first = XmppEventLog [  messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 10 messages_sent_dropped_sending_to_syslog = 0 ],  ] ] tx_msg_agg= [  [ _iter59->first = dropped_client_send_failed _iter59->second = 0,  _iter59->first = dropped_header_write_failed _iter59->second = 0,  _iter59->first = dropped_no_client _iter59->second = 1,  _iter59->first = dropped_no_queue _iter59->second = 0,  _iter59->first = dropped_no_session _iter59->second = 0,  _iter59->first = dropped_queue_level _iter59->second = 0,  _iter59->first = dropped_rate_limited _iter59->second = 0,  _iter59->first = dropped_sending_disabled _iter59->second = 41,  _iter59->first = dropped_sending_to_syslog _iter59->second = 0,  _iter59->first = dropped_session_not_connected _iter59->second = 0,  _iter59->first = dropped_validation_failed _iter59->second = 0,  _iter59->first = dropped_write_failed _iter59->second = 0,  _iter59->first = dropped_wrong_client_sm_state _iter59->second = 67,  _iter59->first = sent _iter59->second = 0,  ] ] ]
2024-10-17 Thu 02:10:52:780.015 UTC  cn-jenkins-deploy-platform-k8s-manifests-101-1 [Thread 140693227857664, Pid 809]: SANDESH: Send FAILED: 1729131052779810  [SYS_NOTICE]: UveVrouterAgent: data= [ name = cn-jenkins-deploy-platform-k8s-manifests-101-1.cluster.local xmpp_peer_list= [  [ [  ip = 10.0.0.33 status = 0 setup_time = 0 primary = 1 ],  ] ] self_ip_list= [  [ (*_iter146) = 10.0.0.33,  ] ] build_info = {"build-info":[{"build-version":"nightly","build-time":"2024-10-17 00:14:03.057316","build-user":"root","build-hostname":"jenkins-build-prepare-sandbox-6040-1.novalocal","build-id":"nightly-nightly.el7","build-number":"@contrail"}]} phy_if= [  [ [  name = eth0 mac_address = fa:16:3e:0f:4a:92 ],  ] ] vhost_if= [  name = vhost0 mac_address = fa:16:3e:0f:4a:92 ] log_file = /var/log/contrail/vrouter-agent/contrail-vrouter-agent.log config_file = /etc/contrail/contrail-vrouter-agent.conf log_local = 1 log_level = SYS_NOTICE log_category =  sandesh_http_port = 8085 dns_servers= [  [ (*_iter149) = 10.0.0.33,  ] ] vhost_cfg= [  name = vhost0 ip = 10.0.0.33 ip_prefix_len = 24 gateway = 10.0.0.1 ] eth_name= [  [ (*_iter150) = eth0,  ] ] tunnel_type = MPLSoGRE hypervisor = kvm control_ip = 10.0.0.33 dns_server_list_cfg= [  [ (*_iter152) = 10.0.0.33:53,  ] ] control_node_list_cfg= [  [ (*_iter153) = 10.0.0.33:5269,  ] ] hostname_cfg = cn-jenkins-deploy-platform-k8s-manifests-101-1.cluster.local flow_cache_timeout_cfg = 0 ll_max_system_flows_cfg = 16384 ll_max_vm_flows_cfg = 16384 max_vm_flows_cfg = 100 gateway_cfg_list= [  [ ] ] headless_mode_cfg = 1 collector_server_list_cfg= [  [ (*_iter155) = 10.0.0.33:8086,  ] ] log_flow = 0 mode = VROUTER platform = HOST vr_limits= [  max_labels = 5120 max_nexthops = 524288 max_interfaces = 4355 max_vrfs = 4096 max_mirror_entries = 255 vrouter_max_bridge_entries = 262144 vrouter_max_oflow_bridge_entries = 53248 vrouter_build_info = {"build-info": [{"build-version": "nightly", "build-time": "2024-10-17 00:11:42.948705", "build-user": "root", "build-hostname": "jenkins-build-prepare-sandbox-6040-1.novalocal"}]} vrouter_max_flow_entries = 524288 vrouter_max_oflow_entries = 105472 vrouter_priority_tagging = 1 ] bgpaas_enabled = 0 port_mirror_enabled = 0 subcluster_name =  vr_high_watermark = 80 vr_low_watermark = 75 vr_limit_exceeded_list= [  [ ] ] res_limit = 0 res_table_limit = 0 loopback_ip = 0.0.0.0 gateway_list= [  [ (*_iter161) = 10.0.0.1,  ] ] ]
2024-10-17 Thu 02:10:52:780.193 UTC  cn-jenkins-deploy-platform-k8s-manifests-101-1 [Thread 140693227857664, Pid 809]: SANDESH: Send FAILED: 1729131052779892  [SYS_NOTICE]: VrouterStats: data= [ name = cn-jenkins-deploy-platform-k8s-manifests-101-1.cluster.local in_tpkts = 20638 in_bytes = 1484728 out_tpkts = 51531 out_bytes = 572350727 exception_packets = 25 exception_packets_dropped = 8 exception_packets_allowed = 17 total_flows = 0 aged_flows = 0 uptime = 1729131022309417 phy_if_info= [  [ _iter254->first = eth0 [  speed = -1 duplexity = 255 ],  ] ] raw_vhost_stats= [  name = vhost0 in_pkts = 20638 in_bytes = 1484728 out_pkts = 51531 out_bytes = 572350727 speed = -1 duplexity = -1 ] raw_vhost_drop_stats= [  ds_discard = 0 ds_pull = 0 ds_invalid_if = 0 ds_invalid_arp = 0 ds_trap_no_if = 0 ds_nowhere_to_go = 0 ds_flow_queue_limit_exceeded = 0 ds_flow_no_memory = 0 ds_flow_invalid_protocol = 0 ds_flow_nat_no_rflow = 0 ds_flow_action_drop = 0 ds_flow_action_invalid = 0 ds_flow_unusable = 0 ds_flow_table_full = 0 ds_interface_tx_discard = 0 ds_interface_drop = 0 ds_duplicated = 0 ds_push = 0 ds_ttl_exceeded = 0 ds_invalid_nh = 1 ds_invalid_label = 0 ds_invalid_protocol = 0 ds_interface_rx_discard = 0 ds_invalid_mcast_source = 0 ds_head_alloc_fail = 0 ds_pcow_fail = 0 ds_mcast_clone_fail = 0 ds_rewrite_fail = 0 ds_misc = 0 ds_invalid_packet = 0 ds_cksum_err = 0 ds_no_fmd = 0 ds_invalid_vnid = 0 ds_frag_err = 0 ds_invalid_source = 0 ds_mcast_df_bit = 0 ds_l2_no_route = 0 ds_vlan_fwd_tx = 0 ds_vlan_fwd_enq = 0 ds_drop_new_flow = 0 ds_flow_evict = 0 ds_trap_original = 0 ds_no_memory = 0 ds_fragment_queue_fail = 0 ds_no_frag_entry = 0 ds_icmp_error = 0 ds_clone_fail = 0 ds_drop_pkts = 1 ] raw_drop_stats= [  ds_discard = 0 ds_pull = 0 ds_invalid_if = 0 ds_invalid_arp = 0 ds_trap_no_if = 0 ds_nowhere_to_go = 0 ds_flow_queue_limit_exceeded = 0 ds_flow_no_memory = 0 ds_flow_invalid_protocol = 0 ds_flow_nat_no_rflow = 0 ds_flow_action_drop = 0 ds_flow_action_invalid = 0 ds_flow_unusable = 0 ds_flow_table_full = 0 ds_interface_tx_discard = 0 ds_interface_drop = 0 ds_duplicated = 0 ds_push = 0 ds_ttl_exceeded = 0 ds_invalid_nh = 1 ds_invalid_label = 0 ds_invalid_protocol = 0 ds_interface_rx_discard = 0 ds_invalid_mcast_source = 0 ds_head_alloc_fail = 0 ds_pcow_fail = 0 ds_mcast_clone_fail = 0 ds_rewrite_fail = 0 ds_misc = 0 ds_invalid_packet = 0 ds_cksum_err = 0 ds_no_fmd = 0 ds_invalid_vnid = 0 ds_frag_err = 0 ds_invalid_source = 0 ds_mcast_df_bit = 0 ds_l2_no_route = 0 ds_vlan_fwd_tx = 0 ds_vlan_fwd_enq = 0 ds_drop_new_flow = 0 ds_flow_evict = 0 ds_trap_original = 0 ds_no_memory = 0 ds_fragment_queue_fail = 0 ds_no_frag_entry = 0 ds_icmp_error = 0 ds_clone_fail = 0 ds_drop_pkts = 1 ] flow_rate= [  added_flows = 0 max_flow_adds_per_second = 0 min_flow_adds_per_second = 0 deleted_flows = 0 max_flow_deletes_per_second = 0 min_flow_deletes_per_second = 0 active_flows = 0 hold_flows = 0 ] raw_phy_if_stats= [  [ _iter262->first = eth0 [  in_pkts = 51506 in_bytes = 572349677 out_pkts = 20639 out_bytes = 1484746 ],  ] ] raw_phy_if_drop_stats= [  [ _iter263->first = eth0 [  ds_discard = 0 ds_pull = 0 ds_invalid_if = 0 ds_invalid_arp = 0 ds_trap_no_if = 0 ds_nowhere_to_go = 0 ds_flow_queue_limit_exceeded = 0 ds_flow_no_memory = 0 ds_flow_invalid_protocol = 0 ds_flow_nat_no_rflow = 0 ds_flow_action_drop = 0 ds_flow_action_invalid = 0 ds_flow_unusable = 0 ds_flow_table_full = 0 ds_interface_tx_discard = 0 ds_interface_drop = 0 ds_duplicated = 0 ds_push = 0 ds_ttl_exceeded = 0 ds_invalid_nh = 0 ds_invalid_label = 0 ds_invalid_protocol = 0 ds_interface_rx_discard = 0 ds_invalid_mcast_source = 0 ds_head_alloc_fail = 0 ds_pcow_fail = 0 ds_mcast_clone_fail = 0 ds_rewrite_fail = 0 ds_misc = 0 ds_invalid_packet = 0 ds_cksum_err = 0 ds_no_fmd = 0 ds_invalid_vnid = 0 ds_frag_err = 0 ds_invalid_source = 0 ds_mcast_df_bit = 0 ds_l2_no_route = 0 ds_vlan_fwd_tx = 0 ds_vlan_fwd_enq = 0 ds_drop_new_flow = 0 ds_flow_evict = 0 ds_trap_original = 0 ds_no_memory = 0 ds_fragment_queue_fail = 0 ds_no_frag_entry = 0 ds_icmp_error = 0 ds_clone_fail = 0 ds_drop_pkts = 0 ],  ] ] ]
2024-10-17 Thu 02:10:52:780.272 UTC  cn-jenkins-deploy-platform-k8s-manifests-101-1 [Thread 140693227857664, Pid 809]: SANDESH: Send FAILED: 1729131052780240  [SYS_NOTICE]: VrouterControlStatsTrace: data= [ name = cn-jenkins-deploy-platform-k8s-manifests-101-1.cluster.local raw_xmpp_stats= [  [ _iter306->first = 10.0.0.33 [  reconnects = 0 in_msgs = 0 out_msgs = 0 ],  ] ] raw_ifmap_stats= [  [ _iter307->first = link_delete_parse_errors _iter307->second = 0,  _iter307->first = link_update_parse_errors _iter307->second = 0,  _iter307->first = node_delete_parse_errors _iter307->second = 0,  _iter307->first = node_update_parse_errors _iter307->second = 0,  ] ] raw_rt_table_size= [  [ _iter308->first = default-domain:default-project:ip-fabric:__default__ [  inet4_unicast = 7 inet4_multicast = 0 evpn = 0 bridge = 5 inet6_unicast = 2 ],  _iter308->first = default-domain:default-project:ip-fabric:ip-fabric [  inet4_unicast = 2 inet4_multicast = 0 evpn = 1 bridge = 2 inet6_unicast = 0 ],  ] ] ]
2024-10-17 Thu 02:10:54:226.906 UTC  cn-jenkins-deploy-platform-k8s-manifests-101-1 [Thread 140693232056064, Pid 809]: SANDESH: Send FAILED: 1729131054226672  [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.0.0.33 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709
2024-10-17 Thu 02:10:54:226.954 UTC  cn-jenkins-deploy-platform-k8s-manifests-101-1 [Thread 140693232056064, Pid 809]: SANDESH: Send FAILED: 1729131054226860  [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.0.0.33 state = Connect event = Connect Event local_port = 40955 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709
2024-10-17 Thu 02:10:54:226.997 UTC  cn-jenkins-deploy-platform-k8s-manifests-101-1 [Thread 140693376776512, Pid 809]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer  RECV  Mode Client:  Event: Tcp Connect Fail  peer ip: 10.0.0.33 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1302
2024-10-17 Thu 02:10:54:227.042 UTC  cn-jenkins-deploy-platform-k8s-manifests-101-1 [Thread 140693215262464, Pid 809]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 314
2024-10-17 Thu 02:10:54:227.120 UTC  cn-jenkins-deploy-platform-k8s-manifests-101-1 [Thread 140693215262464, Pid 809]: SANDESH: Send FAILED: 1729131054227075  [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.0.0.33 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709
2024-10-17 Thu 02:10:54:227.131 UTC  cn-jenkins-deploy-platform-k8s-manifests-101-1 [Thread 140693215262464, Pid 809]: SANDESH: Send FAILED: 1729131054227081  [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.0.0.33 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709
2024-10-17 Thu 02:10:55:347.027 UTC  cn-jenkins-deploy-platform-k8s-manifests-101-1 [Thread 140693223659264, Pid 809]: SANDESH: Send FAILED: 1729131055346730  [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-k8s-manifests-101-1.cluster.local process_status= [  [ [  module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [  [ [  type = XMPP name = control-node:10.0.0.33 server_addrs= [  [ (*_iter6) = 10.0.0.33:5269,  ] ] status = Down description = Connect ],  [  type = XMPP name = dns-server:10.0.0.33 server_addrs= [  [ (*_iter6) = 10.0.0.33:53,  ] ] status = Down description = Connect ],  [  type = Collector name =  server_addrs= [  [ (*_iter6) = 10.0.0.33:8086,  ] ] status = Initializing description = Idle : EvIdleHoldTimerExpired -> Connect ],  ] ] flag_infos= [  [ ] ] description = XMPP:control-node:10.0.0.33, XMPP:dns-server:10.0.0.33, Collector connection down, No Configuration for self ],  ] ] ]
2024-10-17 Thu 02:10:55:347.225 UTC  cn-jenkins-deploy-platform-k8s-manifests-101-1 [Thread 140693223659264, Pid 809]: SANDESH: Send FAILED: 1729131055346993  [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-k8s-manifests-101-1.cluster.local:Compute:contrail-vrouter-agent:0 client_info= [  status = Connect successful_connections = 0 pid = 809 http_port = 8085 start_time = 1729131022563768 collector_name =  collector_ip = 10.0.0.33:8086 collector_list= [  [ (*_iter40) = 10.0.0.33:8086,  ] ] rx_socket_stats= [  bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration =  errors = 0 ] tx_socket_stats= [  bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration =  errors = 0 ] ] msg_type_agg= [  [ _iter58->first = NodeStatusUVE [  messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 15 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ],  _iter58->first = SandeshModuleClientTrace [  messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 14 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ],  _iter58->first = TcpServerMessageLog [  messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ],  _iter58->first = TcpSessionMessageLog [  messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 30 messages_sent_dropped_sending_to_syslog = 0 ],  _iter58->first = UveVirtualNetworkAgentTrace [  messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ],  _iter58->first = UveVrouterAgent [  messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ],  _iter58->first = VersionInfoLog [  messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ],  _iter58->first = VrfObjectLog [  messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ],  _iter58->first = VrouterControlStatsTrace [  messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ],  _iter58->first = VrouterStats [  messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ],  _iter58->first = XmppConnectionLog [  messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 40 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ],  _iter58->first = XmppEventLog [  messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 11 messages_sent_dropped_sending_to_syslog = 0 ],  ] ] tx_msg_agg= [  [ _iter59->first = dropped_client_send_failed _iter59->second = 0,  _iter59->first = dropped_header_write_failed _iter59->second = 0,  _iter59->first = dropped_no_client _iter59->second = 1,  _iter59->first = dropped_no_queue _iter59->second = 0,  _iter59->first = dropped_no_session _iter59->second = 0,  _iter59->first = dropped_queue_level _iter59->second = 0,  _iter59->first = dropped_rate_limited _iter59->second = 0,  _iter59->first = dropped_sending_disabled _iter59->second = 45,  _iter59->first = dropped_sending_to_syslog _iter59->second = 0,  _iter59->first = dropped_session_not_connected _iter59->second = 0,  _iter59->first = dropped_validation_failed _iter59->second = 0,  _iter59->first = dropped_write_failed _iter59->second = 0,  _iter59->first = dropped_wrong_client_sm_state _iter59->second = 76,  _iter59->first = sent _iter59->second = 0,  ] ] ]
2024-10-17 Thu 02:10:55:347.334 UTC  cn-jenkins-deploy-platform-k8s-manifests-101-1 [Thread 140693227857664, Pid 809]: TCP [SYS_ERR]: TcpSessionMessageLog: Session -(-1) > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 314
2024-10-17 Thu 02:10:55:347.425 UTC  cn-jenkins-deploy-platform-k8s-manifests-101-1 [Thread 140693227857664, Pid 809]: SANDESH: Send FAILED: 1729131055347320  [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-k8s-manifests-101-1.cluster.local process_status= [  [ [  module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [  [ [  type = XMPP name = control-node:10.0.0.33 server_addrs= [  [ (*_iter6) = 10.0.0.33:5269,  ] ] status = Down description = Connect ],  [  type = XMPP name = dns-server:10.0.0.33 server_addrs= [  [ (*_iter6) = 10.0.0.33:53,  ] ] status = Down description = Connect ],  [  type = Collector name =  server_addrs= [  [ (*_iter6) = 0.0.0.0:0,  ] ] status = Down description = Connect : EvTcpConnectFail ],  ] ] flag_infos= [  [ ] ] description = XMPP:control-node:10.0.0.33, XMPP:dns-server:10.0.0.33, Collector connection down, No Configuration for self ],  ] ] ]
2024-10-17 Thu 02:10:55:347.610 UTC  cn-jenkins-deploy-platform-k8s-manifests-101-1 [Thread 140693227857664, Pid 809]: SANDESH: Send FAILED: 1729131055347403  [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-k8s-manifests-101-1.cluster.local:Compute:contrail-vrouter-agent:0 client_info= [  status = Idle successful_connections = 0 pid = 809 http_port = 8085 start_time = 1729131022563768 collector_name =  collector_ip = 10.0.0.33:8086 collector_list= [  [ (*_iter40) = 10.0.0.33:8086,  ] ] rx_socket_stats= [  bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration =  errors = 0 ] tx_socket_stats= [  bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration =  errors = 0 ] ] msg_type_agg= [  [ _iter58->first = NodeStatusUVE [  messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 16 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ],  _iter58->first = SandeshModuleClientTrace [  messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 15 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ],  _iter58->first = TcpServerMessageLog [  messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ],  _iter58->first = TcpSessionMessageLog [  messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 31 messages_sent_dropped_sending_to_syslog = 0 ],  _iter58->first = UveVirtualNetworkAgentTrace [  messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ],  _iter58->first = UveVrouterAgent [  messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ],  _iter58->first = VersionInfoLog [  messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ],  _iter58->first = VrfObjectLog [  messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ],  _iter58->first = VrouterControlStatsTrace [  messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ],  _iter58->first = VrouterStats [  messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ],  _iter58->first = XmppConnectionLog [  messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 40 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ],  _iter58->first = XmppEventLog [  messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 11 messages_sent_dropped_sending_to_syslog = 0 ],  ] ] tx_msg_agg= [  [ _iter59->first = dropped_client_send_failed _iter59->second = 0,  _iter59->first = dropped_header_write_failed _iter59->second = 0,  _iter59->first = dropped_no_client _iter59->second = 1,  _iter59->first = dropped_no_queue _iter59->second = 0,  _iter59->first = dropped_no_session _iter59->second = 0,  _iter59->first = dropped_queue_level _iter59->second = 0,  _iter59->first = dropped_rate_limited _iter59->second = 0,  _iter59->first = dropped_sending_disabled _iter59->second = 46,  _iter59->first = dropped_sending_to_syslog _iter59->second = 0,  _iter59->first = dropped_session_not_connected _iter59->second = 0,  _iter59->first = dropped_validation_failed _iter59->second = 0,  _iter59->first = dropped_write_failed _iter59->second = 0,  _iter59->first = dropped_wrong_client_sm_state _iter59->second = 78,  _iter59->first = sent _iter59->second = 0,  ] ] ]
2024-10-17 Thu 02:10:56:745.761 UTC  cn-jenkins-deploy-platform-k8s-manifests-101-1 [Thread 140693232056064, Pid 809]: SANDESH: Send FAILED: 1729131056745654  [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.0.0.33 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709
2024-10-17 Thu 02:10:56:746.081 UTC  cn-jenkins-deploy-platform-k8s-manifests-101-1 [Thread 140693376776512, Pid 809]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer  RECV  Mode Client:  Event: Tcp Connect Fail  peer ip: 10.0.0.33 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1302
2024-10-17 Thu 02:10:56:746.134 UTC  cn-jenkins-deploy-platform-k8s-manifests-101-1 [Thread 140693227857664, Pid 809]: SANDESH: Send FAILED: 1729131056746103  [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.0.0.33 state = Connect event = Connect Event local_port = 44051 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709
2024-10-17 Thu 02:10:56:746.327 UTC  cn-jenkins-deploy-platform-k8s-manifests-101-1 [Thread 140693215262464, Pid 809]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 314
2024-10-17 Thu 02:10:56:746.402 UTC  cn-jenkins-deploy-platform-k8s-manifests-101-1 [Thread 140693211064064, Pid 809]: SANDESH: Send FAILED: 1729131056746349  [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.0.0.33 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709
2024-10-17 Thu 02:10:56:746.416 UTC  cn-jenkins-deploy-platform-k8s-manifests-101-1 [Thread 140693211064064, Pid 809]: SANDESH: Send FAILED: 1729131056746361  [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.0.0.33 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709
2024-10-17 Thu 02:11:22:568.797 UTC  cn-jenkins-deploy-platform-k8s-manifests-101-1 [Thread 140693219460864, Pid 809]: SANDESH: Send FAILED: 1729131082567649  [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-k8s-manifests-101-1.cluster.local:Compute:contrail-vrouter-agent:0 sm_queue_count = 0 sm_stats= [  ev_stats= [  [ [  event = scm::EvCollectorUpdate enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ],  [  event = scm::EvIdleHoldTimerExpired enqueues = 5 dequeues = 5 enqueue_fails = 0 dequeue_fails = 0 ],  [  event = scm::EvSandeshSend enqueues = 84 dequeues = 84 enqueue_fails = 0 dequeue_fails = 0 ],  [  event = scm::EvStart enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ],  [  event = scm::EvTcpConnectFail enqueues = 6 dequeues = 6 enqueue_fails = 0 dequeue_fails = 0 ],  [  event = scm::EvTcpDeleteSession enqueues = 6 dequeues = 6 enqueue_fails = 0 dequeue_fails = 0 ],  [  event =  enqueues = 103 dequeues = 103 enqueue_fails = 0 dequeue_fails = 0 ],  ] ] state = Idle state_since = 1729131055347360 last_state = Idle last_event = scm::EvSandeshSend last_event_at = 1729131056746412 ] max_sm_queue_count = 7 ]
2024-10-17 Thu 02:11:22:569.023 UTC  cn-jenkins-deploy-platform-k8s-manifests-101-1 [Thread 140693219460864, Pid 809]: SANDESH: Send FAILED: 1729131082568044  [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-k8s-manifests-101-1.cluster.local:Compute:contrail-vrouter-agent:0 client_info= [  status = Idle successful_connections = 0 pid = 809 http_port = 8085 start_time = 1729131022563768 collector_name =  collector_ip = 10.0.0.33:8086 collector_list= [  [ (*_iter40) = 10.0.0.33:8086,  ] ] rx_socket_stats= [  bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration =  errors = 0 ] tx_socket_stats= [  bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration =  errors = 0 ] ] msg_type_agg= [  [ _iter58->first = NodeStatusUVE [  messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 17 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ],  _iter58->first = SandeshModuleClientTrace [  messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 16 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ],  _iter58->first = TcpServerMessageLog [  messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ],  _iter58->first = TcpSessionMessageLog [  messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 42 messages_sent_dropped_sending_to_syslog = 0 ],  _iter58->first = UveVirtualNetworkAgentTrace [  messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ],  _iter58->first = UveVrouterAgent [  messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ],  _iter58->first = VersionInfoLog [  messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ],  _iter58->first = VrfObjectLog [  messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ],  _iter58->first = VrouterControlStatsTrace [  messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ],  _iter58->first = VrouterStats [  messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ],  _iter58->first = XmppConnectionLog [  messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 44 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ],  _iter58->first = XmppEventLog [  messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 12 messages_sent_dropped_sending_to_syslog = 0 ],  ] ] tx_msg_agg= [  [ _iter59->first = dropped_client_send_failed _iter59->second = 0,  _iter59->first = dropped_header_write_failed _iter59->second = 0,  _iter59->first = dropped_no_client _iter59->second = 1,  _iter59->first = dropped_no_queue _iter59->second = 0,  _iter59->first = dropped_no_session _iter59->second = 0,  _iter59->first = dropped_queue_level _iter59->second = 0,  _iter59->first = dropped_rate_limited _iter59->second = 0,  _iter59->first = dropped_sending_disabled _iter59->second = 58,  _iter59->first = dropped_sending_to_syslog _iter59->second = 0,  _iter59->first = dropped_session_not_connected _iter59->second = 0,  _iter59->first = dropped_validation_failed _iter59->second = 0,  _iter59->first = dropped_write_failed _iter59->second = 0,  _iter59->first = dropped_wrong_client_sm_state _iter59->second = 84,  _iter59->first = sent _iter59->second = 0,  ] ] ]
2024-10-17 Thu 02:11:22:780.746 UTC  cn-jenkins-deploy-platform-k8s-manifests-101-1 [Thread 140693232056064, Pid 809]: SANDESH: Send FAILED: 1729131082780613  [SYS_NOTICE]: VrouterStats: data= [ name = cn-jenkins-deploy-platform-k8s-manifests-101-1.cluster.local cpu_info= [  num_cpu = 4 sys_mem_info= [  total = 25638656 used = 0 free = 3101617710 buffers = 32757 cached = 25502368 node_type =  ] meminfo= [  virt = 32757 peakvirt = 3087546776 res = 32757 ] cpuload= [  one_min_avg = 7.2225 five_min_avg = 2.3075 fifteen_min_avg = 0.8375 ] cpu_share = 0.364601 ] ]
2024-10-17 Thu 02:11:22:780.793 UTC  cn-jenkins-deploy-platform-k8s-manifests-101-1 [Thread 140693232056064, Pid 809]: SANDESH: Send FAILED: 1729131082780632  [SYS_NOTICE]: ComputeCpuStateTrace: data= [ name = cn-jenkins-deploy-platform-k8s-manifests-101-1.cluster.local cpu_info= [  [ [  mem_virt = 32757 cpu_share = 0.364601 used_sys_mem = 4269432171 one_min_cpuload = 7.2225 mem_res = 32757 ],  ] ] ]
2024-10-17 Thu 02:11:22:780.925 UTC  cn-jenkins-deploy-platform-k8s-manifests-101-1 [Thread 140693232056064, Pid 809]: SANDESH: Send FAILED: 1729131082780671  [SYS_NOTICE]: VrouterStats: data= [ name = cn-jenkins-deploy-platform-k8s-manifests-101-1.cluster.local in_tpkts = 35487 in_bytes = 2555210 out_tpkts = 87294 out_bytes = 965153403 exception_packets = 27 exception_packets_dropped = 9 exception_packets_allowed = 18 raw_vhost_stats= [  name = vhost0 in_pkts = 35487 in_bytes = 2555210 out_pkts = 87294 out_bytes = 965153403 speed = -1 duplexity = -1 ] raw_vhost_drop_stats= [  ds_discard = 0 ds_pull = 0 ds_invalid_if = 0 ds_invalid_arp = 0 ds_trap_no_if = 0 ds_nowhere_to_go = 0 ds_flow_queue_limit_exceeded = 0 ds_flow_no_memory = 0 ds_flow_invalid_protocol = 0 ds_flow_nat_no_rflow = 0 ds_flow_action_drop = 0 ds_flow_action_invalid = 0 ds_flow_unusable = 0 ds_flow_table_full = 0 ds_interface_tx_discard = 0 ds_interface_drop = 38 ds_duplicated = 0 ds_push = 0 ds_ttl_exceeded = 0 ds_invalid_nh = 1 ds_invalid_label = 0 ds_invalid_protocol = 0 ds_interface_rx_discard = 0 ds_invalid_mcast_source = 0 ds_head_alloc_fail = 0 ds_pcow_fail = 0 ds_mcast_clone_fail = 0 ds_rewrite_fail = 0 ds_misc = 0 ds_invalid_packet = 0 ds_cksum_err = 0 ds_no_fmd = 0 ds_invalid_vnid = 0 ds_frag_err = 0 ds_invalid_source = 0 ds_mcast_df_bit = 0 ds_l2_no_route = 0 ds_vlan_fwd_tx = 0 ds_vlan_fwd_enq = 0 ds_drop_new_flow = 0 ds_flow_evict = 0 ds_trap_original = 0 ds_no_memory = 0 ds_fragment_queue_fail = 0 ds_no_frag_entry = 0 ds_icmp_error = 0 ds_clone_fail = 0 ds_drop_pkts = 39 ] raw_drop_stats= [  ds_discard = 0 ds_pull = 0 ds_invalid_if = 0 ds_invalid_arp = 0 ds_trap_no_if = 0 ds_nowhere_to_go = 0 ds_flow_queue_limit_exceeded = 0 ds_flow_no_memory = 0 ds_flow_invalid_protocol = 0 ds_flow_nat_no_rflow = 0 ds_flow_action_drop = 0 ds_flow_action_invalid = 0 ds_flow_unusable = 0 ds_flow_table_full = 0 ds_interface_tx_discard = 0 ds_interface_drop = 38 ds_duplicated = 0 ds_push = 0 ds_ttl_exceeded = 0 ds_invalid_nh = 1 ds_invalid_label = 0 ds_invalid_protocol = 0 ds_interface_rx_discard = 0 ds_invalid_mcast_source = 0 ds_head_alloc_fail = 0 ds_pcow_fail = 0 ds_mcast_clone_fail = 0 ds_rewrite_fail = 0 ds_misc = 0 ds_invalid_packet = 0 ds_cksum_err = 0 ds_no_fmd = 0 ds_invalid_vnid = 0 ds_frag_err = 0 ds_invalid_source = 0 ds_mcast_df_bit = 0 ds_l2_no_route = 0 ds_vlan_fwd_tx = 0 ds_vlan_fwd_enq = 0 ds_drop_new_flow = 0 ds_flow_evict = 0 ds_trap_original = 0 ds_no_memory = 0 ds_fragment_queue_fail = 0 ds_no_frag_entry = 0 ds_icmp_error = 0 ds_clone_fail = 0 ds_drop_pkts = 39 ] flow_rate= [  added_flows = 0 max_flow_adds_per_second = 0 min_flow_adds_per_second = 0 deleted_flows = 0 max_flow_deletes_per_second = 0 min_flow_deletes_per_second = 0 active_flows = 0 hold_flows = 0 ] raw_phy_if_stats= [  [ _iter262->first = eth0 [  in_pkts = 87254 in_bytes = 965151723 out_pkts = 35488 out_bytes = 2555228 ],  ] ] raw_phy_if_drop_stats= [  [ _iter263->first = eth0 [  ds_discard = 0 ds_pull = 0 ds_invalid_if = 0 ds_invalid_arp = 0 ds_trap_no_if = 0 ds_nowhere_to_go = 0 ds_flow_queue_limit_exceeded = 0 ds_flow_no_memory = 0 ds_flow_invalid_protocol = 0 ds_flow_nat_no_rflow = 0 ds_flow_action_drop = 0 ds_flow_action_invalid = 0 ds_flow_unusable = 0 ds_flow_table_full = 0 ds_interface_tx_discard = 0 ds_interface_drop = 38 ds_duplicated = 0 ds_push = 0 ds_ttl_exceeded = 0 ds_invalid_nh = 0 ds_invalid_label = 0 ds_invalid_protocol = 0 ds_interface_rx_discard = 0 ds_invalid_mcast_source = 0 ds_head_alloc_fail = 0 ds_pcow_fail = 0 ds_mcast_clone_fail = 0 ds_rewrite_fail = 0 ds_misc = 0 ds_invalid_packet = 0 ds_cksum_err = 0 ds_no_fmd = 0 ds_invalid_vnid = 0 ds_frag_err = 0 ds_invalid_source = 0 ds_mcast_df_bit = 0 ds_l2_no_route = 0 ds_vlan_fwd_tx = 0 ds_vlan_fwd_enq = 0 ds_drop_new_flow = 0 ds_flow_evict = 0 ds_trap_original = 0 ds_no_memory = 0 ds_fragment_queue_fail = 0 ds_no_frag_entry = 0 ds_icmp_error = 0 ds_clone_fail = 0 ds_drop_pkts = 38 ],  ] ] ]
2024-10-17 Thu 02:11:22:780.961 UTC  cn-jenkins-deploy-platform-k8s-manifests-101-1 [Thread 140693232056064, Pid 809]: SANDESH: Send FAILED: 1729131082780701  [SYS_NOTICE]: VrouterControlStatsTrace: data= [ name = cn-jenkins-deploy-platform-k8s-manifests-101-1.cluster.local raw_xmpp_stats= [  [ _iter306->first = 10.0.0.33 [  reconnects = 0 in_msgs = 0 out_msgs = 0 ],  ] ] raw_ifmap_stats= [  [ _iter307->first = link_delete_parse_errors _iter307->second = 0,  _iter307->first = link_update_parse_errors _iter307->second = 0,  _iter307->first = node_delete_parse_errors _iter307->second = 0,  _iter307->first = node_update_parse_errors _iter307->second = 0,  ] ] raw_rt_table_size= [  [ _iter308->first = default-domain:default-project:ip-fabric:__default__ [  inet4_unicast = 7 inet4_multicast = 0 evpn = 0 bridge = 5 inet6_unicast = 2 ],  _iter308->first = default-domain:default-project:ip-fabric:ip-fabric [  inet4_unicast = 2 inet4_multicast = 0 evpn = 1 bridge = 2 inet6_unicast = 0 ],  ] ] ]
2024-10-17 Thu 02:11:24:016.757 UTC  cn-jenkins-deploy-platform-k8s-manifests-101-1 [Thread 140693232056064, Pid 809]: SANDESH: Send FAILED: 1729131084016653  [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.0.0.33 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709
2024-10-17 Thu 02:11:24:017.619 UTC  cn-jenkins-deploy-platform-k8s-manifests-101-1 [Thread 140693376776512, Pid 809]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer  RECV  Mode Client:  Event: Tcp Connect Fail  peer ip: 10.0.0.33 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1302
2024-10-17 Thu 02:11:24:017.679 UTC  cn-jenkins-deploy-platform-k8s-manifests-101-1 [Thread 140693211064064, Pid 809]: SANDESH: Send FAILED: 1729131084017646  [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.0.0.33 state = Connect event = Connect Event local_port = 35369 remote_port = 4294967295 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709
2024-10-17 Thu 02:11:24:017.961 UTC  cn-jenkins-deploy-platform-k8s-manifests-101-1 [Thread 140693215262464, Pid 809]: TCP [SYS_ERR]: TcpSessionMessageLog: Session - > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 314
2024-10-17 Thu 02:11:24:018.022 UTC  cn-jenkins-deploy-platform-k8s-manifests-101-1 [Thread 140693223659264, Pid 809]: SANDESH: Send FAILED: 1729131084017981  [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.0.0.33 state = Connect next_state = Active event = EvTcpConnectFail ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709
2024-10-17 Thu 02:11:24:018.034 UTC  cn-jenkins-deploy-platform-k8s-manifests-101-1 [Thread 140693223659264, Pid 809]: SANDESH: Send FAILED: 1729131084017985  [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.0.0.33 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709
2024-10-17 Thu 02:11:24:197.271 UTC  cn-jenkins-deploy-platform-k8s-manifests-101-1 [Thread 140693211064064, Pid 809]: SANDESH: Send FAILED: 1729131084197177  [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.0.0.33 state = Active next_state = Connect event = EvConnectTimerExpired ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709
2024-10-17 Thu 02:11:24:200.219 UTC  cn-jenkins-deploy-platform-k8s-manifests-101-1 [Thread 140693376776512, Pid 809]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer cn-jenkins-deploy-platform-k8s-manifests-101-1.cluster.local:10.0.0.33 RECV  Mode Client:  Event: Tcp Connected  peer ip: 10.0.0.33 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1295
2024-10-17 Thu 02:11:24:200.389 UTC  cn-jenkins-deploy-platform-k8s-manifests-101-1 [Thread 140693215262464, Pid 809]: SANDESH: Send FAILED: 1729131084200266  [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.0.0.33 state = Connect event = Connect Event local_port = 46743 remote_port = 5269 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709
2024-10-17 Thu 02:11:24:200.409 UTC  cn-jenkins-deploy-platform-k8s-manifests-101-1 [Thread 140693215262464, Pid 809]: SANDESH: Send FAILED: 1729131084200370  [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.0.0.33 state = Connect next_state = OpenSent event = EvTcpConnected local_port = 46743 remote_port = 5269 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709
2024-10-17 Thu 02:11:24:207.978 UTC  cn-jenkins-deploy-platform-k8s-manifests-101-1 [Thread 140693376776512, Pid 809]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer cn-jenkins-deploy-platform-k8s-manifests-101-1.cluster.local:10.0.0.33 RECV  Mode Client:  Event: Tcp Connection Closed  peer ip: 10.0.0.33 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1310
2024-10-17 Thu 02:11:24:208.136 UTC  cn-jenkins-deploy-platform-k8s-manifests-101-1 [Thread 140693211064064, Pid 809]: SANDESH: Send FAILED: 1729131084208039  [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.0.0.33 state = OpenSent next_state = Established event = EvXmppOpen identifier = cn-jenkins-deploy-platform-k8s-manifests-101-1.cluster.local ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709
2024-10-17 Thu 02:11:24:208.152 UTC  cn-jenkins-deploy-platform-k8s-manifests-101-1 [Thread 140693211064064, Pid 809]: SANDESH: Send FAILED: 1729131084208069  [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.0.0.33 state = Established next_state = Active event = EvTcpClose ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709
2024-10-17 Thu 02:11:24:208.165 UTC  cn-jenkins-deploy-platform-k8s-manifests-101-1 [Thread 140693211064064, Pid 809]: SANDESH: Send FAILED: 1729131084208072  [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.0.0.33 state = Active event = Connect failed after retries ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709
2024-10-17 Thu 02:11:24:208.377 UTC  cn-jenkins-deploy-platform-k8s-manifests-101-1 [Thread 140693202667264, Pid 809]: SANDESH: Send FAILED: 1729131084208267  [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-k8s-manifests-101-1.cluster.local process_status= [  [ [  module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [  [ [  type = XMPP name = control-node:10.0.0.33 server_addrs= [  [ (*_iter6) = 10.0.0.33:5269,  ] ] status = Up description = Established ],  [  type = XMPP name = dns-server:10.0.0.33 server_addrs= [  [ (*_iter6) = 10.0.0.33:53,  ] ] status = Down description = Connect ],  [  type = Collector name =  server_addrs= [  [ (*_iter6) = 0.0.0.0:0,  ] ] status = Down description = Connect : EvTcpConnectFail ],  ] ] flag_infos= [  [ ] ] description = XMPP:dns-server:10.0.0.33, Collector connection down, No Configuration for self ],  ] ] ]
2024-10-17 Thu 02:11:24:209.761 UTC  cn-jenkins-deploy-platform-k8s-manifests-101-1 [Thread 140693211064064, Pid 809]: SANDESH: Send FAILED: 1729131084209700  [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-k8s-manifests-101-1.cluster.local process_status= [  [ [  module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [  [ [  type = XMPP name = control-node:10.0.0.33 server_addrs= [  [ (*_iter6) = 10.0.0.33:5269,  ] ] status = Down description = Established ],  [  type = XMPP name = dns-server:10.0.0.33 server_addrs= [  [ (*_iter6) = 10.0.0.33:53,  ] ] status = Down description = Connect ],  [  type = Collector name =  server_addrs= [  [ (*_iter6) = 0.0.0.0:0,  ] ] status = Down description = Connect : EvTcpConnectFail ],  ] ] flag_infos= [  [ ] ] description = XMPP:control-node:10.0.0.33, XMPP:dns-server:10.0.0.33, Collector connection down, No Configuration for self ],  ] ] ]
2024-10-17 Thu 02:11:25:349.131 UTC  cn-jenkins-deploy-platform-k8s-manifests-101-1 [Thread 140693232056064, Pid 809]: SANDESH: Send FAILED: 1729131085347499  [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-k8s-manifests-101-1.cluster.local process_status= [  [ [  module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [  [ [  type = XMPP name = control-node:10.0.0.33 server_addrs= [  [ (*_iter6) = 10.0.0.33:5269,  ] ] status = Down description = Established ],  [  type = XMPP name = dns-server:10.0.0.33 server_addrs= [  [ (*_iter6) = 10.0.0.33:53,  ] ] status = Down description = Connect ],  [  type = Collector name =  server_addrs= [  [ (*_iter6) = 10.0.0.33:8086,  ] ] status = Initializing description = Idle : EvIdleHoldTimerExpired -> Connect ],  ] ] flag_infos= [  [ ] ] description = XMPP:control-node:10.0.0.33, XMPP:dns-server:10.0.0.33, Collector connection down, No Configuration for self ],  ] ] ]
2024-10-17 Thu 02:11:25:349.412 UTC  cn-jenkins-deploy-platform-k8s-manifests-101-1 [Thread 140693232056064, Pid 809]: TCP [SYS_ERR]: TcpSessionMessageLog: Session -(-1) > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 314
2024-10-17 Thu 02:11:25:350.028 UTC  cn-jenkins-deploy-platform-k8s-manifests-101-1 [Thread 140693232056064, Pid 809]: SANDESH: Send FAILED: 1729131085349077  [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-k8s-manifests-101-1.cluster.local:Compute:contrail-vrouter-agent:0 client_info= [  status = Connect successful_connections = 0 pid = 809 http_port = 8085 start_time = 1729131022563768 collector_name =  collector_ip = 10.0.0.33:8086 collector_list= [  [ (*_iter40) = 10.0.0.33:8086,  ] ] rx_socket_stats= [  bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration =  errors = 0 ] tx_socket_stats= [  bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration =  errors = 0 ] ] msg_type_agg= [  [ _iter58->first = ComputeCpuStateTrace [  messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ],  _iter58->first = NodeStatusUVE [  messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 19 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ],  _iter58->first = SandeshModuleClientTrace [  messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 18 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ],  _iter58->first = TcpServerMessageLog [  messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ],  _iter58->first = TcpSessionMessageLog [  messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 46 messages_sent_dropped_sending_to_syslog = 0 ],  _iter58->first = UveVirtualNetworkAgentTrace [  messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ],  _iter58->first = UveVrouterAgent [  messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ],  _iter58->first = VersionInfoLog [  messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ],  _iter58->first = VrfObjectLog [  messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ],  _iter58->first = VrouterControlStatsTrace [  messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ],  _iter58->first = VrouterStats [  messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 3 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ],  _iter58->first = XmppConnectionLog [  messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 54 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ],  _iter58->first = XmppEventLog [  messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 15 messages_sent_dropped_sending_to_syslog = 0 ],  ] ] tx_msg_agg= [  [ _iter59->first = dropped_client_send_failed _iter59->second = 0,  _iter59->first = dropped_header_write_failed _iter59->second = 0,  _iter59->first = dropped_no_client _iter59->second = 1,  _iter59->first = dropped_no_queue _iter59->second = 0,  _iter59->first = dropped_no_session _iter59->second = 0,  _iter59->first = dropped_queue_level _iter59->second = 0,  _iter59->first = dropped_rate_limited _iter59->second = 0,  _iter59->first = dropped_sending_disabled _iter59->second = 65,  _iter59->first = dropped_sending_to_syslog _iter59->second = 0,  _iter59->first = dropped_session_not_connected _iter59->second = 0,  _iter59->first = dropped_validation_failed _iter59->second = 0,  _iter59->first = dropped_write_failed _iter59->second = 0,  _iter59->first = dropped_wrong_client_sm_state _iter59->second = 102,  _iter59->first = sent _iter59->second = 0,  ] ] ]
2024-10-17 Thu 02:11:25:350.115 UTC  cn-jenkins-deploy-platform-k8s-manifests-101-1 [Thread 140693232056064, Pid 809]: SANDESH: Send FAILED: 1729131085349369  [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-k8s-manifests-101-1.cluster.local process_status= [  [ [  module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [  [ [  type = XMPP name = control-node:10.0.0.33 server_addrs= [  [ (*_iter6) = 10.0.0.33:5269,  ] ] status = Down description = Established ],  [  type = XMPP name = dns-server:10.0.0.33 server_addrs= [  [ (*_iter6) = 10.0.0.33:53,  ] ] status = Down description = Connect ],  [  type = Collector name =  server_addrs= [  [ (*_iter6) = 0.0.0.0:0,  ] ] status = Down description = Connect : EvTcpConnectFail ],  ] ] flag_infos= [  [ ] ] description = XMPP:control-node:10.0.0.33, XMPP:dns-server:10.0.0.33, Collector connection down, No Configuration for self ],  ] ] ]
2024-10-17 Thu 02:11:25:350.437 UTC  cn-jenkins-deploy-platform-k8s-manifests-101-1 [Thread 140693232056064, Pid 809]: SANDESH: Send FAILED: 1729131085349528  [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-k8s-manifests-101-1.cluster.local:Compute:contrail-vrouter-agent:0 client_info= [  status = Idle successful_connections = 0 pid = 809 http_port = 8085 start_time = 1729131022563768 collector_name =  collector_ip = 10.0.0.33:8086 collector_list= [  [ (*_iter40) = 10.0.0.33:8086,  ] ] rx_socket_stats= [  bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration =  errors = 0 ] tx_socket_stats= [  bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration =  errors = 0 ] ] msg_type_agg= [  [ _iter58->first = ComputeCpuStateTrace [  messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ],  _iter58->first = NodeStatusUVE [  messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 20 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ],  _iter58->first = SandeshModuleClientTrace [  messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 18 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ],  _iter58->first = TcpServerMessageLog [  messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ],  _iter58->first = TcpSessionMessageLog [  messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 47 messages_sent_dropped_sending_to_syslog = 0 ],  _iter58->first = UveVirtualNetworkAgentTrace [  messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ],  _iter58->first = UveVrouterAgent [  messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ],  _iter58->first = VersionInfoLog [  messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ],  _iter58->first = VrfObjectLog [  messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ],  _iter58->first = VrouterControlStatsTrace [  messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ],  _iter58->first = VrouterStats [  messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 3 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ],  _iter58->first = XmppConnectionLog [  messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 54 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ],  _iter58->first = XmppEventLog [  messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 15 messages_sent_dropped_sending_to_syslog = 0 ],  ] ] tx_msg_agg= [  [ _iter59->first = dropped_client_send_failed _iter59->second = 0,  _iter59->first = dropped_header_write_failed _iter59->second = 0,  _iter59->first = dropped_no_client _iter59->second = 1,  _iter59->first = dropped_no_queue _iter59->second = 0,  _iter59->first = dropped_no_session _iter59->second = 0,  _iter59->first = dropped_queue_level _iter59->second = 0,  _iter59->first = dropped_rate_limited _iter59->second = 0,  _iter59->first = dropped_sending_disabled _iter59->second = 66,  _iter59->first = dropped_sending_to_syslog _iter59->second = 0,  _iter59->first = dropped_session_not_connected _iter59->second = 0,  _iter59->first = dropped_validation_failed _iter59->second = 0,  _iter59->first = dropped_write_failed _iter59->second = 0,  _iter59->first = dropped_wrong_client_sm_state _iter59->second = 103,  _iter59->first = sent _iter59->second = 0,  ] ] ]
2024-10-17 Thu 02:11:52:568.334 UTC  cn-jenkins-deploy-platform-k8s-manifests-101-1 [Thread 140693223659264, Pid 809]: SANDESH: Send FAILED: 1729131112568136  [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-k8s-manifests-101-1.cluster.local:Compute:contrail-vrouter-agent:0 sm_queue_count = 0 sm_stats= [  ev_stats= [  [ [  event = scm::EvCollectorUpdate enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ],  [  event = scm::EvIdleHoldTimerExpired enqueues = 6 dequeues = 6 enqueue_fails = 0 dequeue_fails = 0 ],  [  event = scm::EvSandeshSend enqueues = 106 dequeues = 106 enqueue_fails = 0 dequeue_fails = 0 ],  [  event = scm::EvStart enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ],  [  event = scm::EvTcpConnectFail enqueues = 7 dequeues = 7 enqueue_fails = 0 dequeue_fails = 0 ],  [  event = scm::EvTcpDeleteSession enqueues = 7 dequeues = 7 enqueue_fails = 0 dequeue_fails = 0 ],  [  event =  enqueues = 128 dequeues = 128 enqueue_fails = 0 dequeue_fails = 0 ],  ] ] state = Idle state_since = 1729131085349464 last_state = Idle last_event = scm::EvSandeshSend last_event_at = 1729131085350152 ] max_sm_queue_count = 7 ]
2024-10-17 Thu 02:11:52:568.535 UTC  cn-jenkins-deploy-platform-k8s-manifests-101-1 [Thread 140693223659264, Pid 809]: SANDESH: Send FAILED: 1729131112568244  [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-k8s-manifests-101-1.cluster.local:Compute:contrail-vrouter-agent:0 client_info= [  status = Idle successful_connections = 0 pid = 809 http_port = 8085 start_time = 1729131022563768 collector_name =  collector_ip = 10.0.0.33:8086 collector_list= [  [ (*_iter40) = 10.0.0.33:8086,  ] ] rx_socket_stats= [  bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration =  errors = 0 ] tx_socket_stats= [  bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration =  errors = 0 ] ] msg_type_agg= [  [ _iter58->first = ComputeCpuStateTrace [  messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ],  _iter58->first = NodeStatusUVE [  messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 21 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ],  _iter58->first = SandeshModuleClientTrace [  messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 20 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ],  _iter58->first = TcpServerMessageLog [  messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ],  _iter58->first = TcpSessionMessageLog [  messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 51 messages_sent_dropped_sending_to_syslog = 0 ],  _iter58->first = UveVirtualNetworkAgentTrace [  messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ],  _iter58->first = UveVrouterAgent [  messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ],  _iter58->first = VersionInfoLog [  messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ],  _iter58->first = VrfObjectLog [  messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ],  _iter58->first = VrouterControlStatsTrace [  messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ],  _iter58->first = VrouterStats [  messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 3 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ],  _iter58->first = XmppConnectionLog [  messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 54 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ],  _iter58->first = XmppEventLog [  messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 15 messages_sent_dropped_sending_to_syslog = 0 ],  ] ] tx_msg_agg= [  [ _iter59->first = dropped_client_send_failed _iter59->second = 0,  _iter59->first = dropped_header_write_failed _iter59->second = 0,  _iter59->first = dropped_no_client _iter59->second = 1,  _iter59->first = dropped_no_queue _iter59->second = 0,  _iter59->first = dropped_no_session _iter59->second = 0,  _iter59->first = dropped_queue_level _iter59->second = 0,  _iter59->first = dropped_rate_limited _iter59->second = 0,  _iter59->first = dropped_sending_disabled _iter59->second = 70,  _iter59->first = dropped_sending_to_syslog _iter59->second = 0,  _iter59->first = dropped_session_not_connected _iter59->second = 0,  _iter59->first = dropped_validation_failed _iter59->second = 0,  _iter59->first = dropped_write_failed _iter59->second = 0,  _iter59->first = dropped_wrong_client_sm_state _iter59->second = 106,  _iter59->first = sent _iter59->second = 0,  ] ] ]
2024-10-17 Thu 02:11:52:780.816 UTC  cn-jenkins-deploy-platform-k8s-manifests-101-1 [Thread 140693219460864, Pid 809]: SANDESH: Send FAILED: 1729131112780724  [SYS_NOTICE]: UveVrouterAgent: data= [ name = cn-jenkins-deploy-platform-k8s-manifests-101-1.cluster.local xmpp_peer_list= [  [ [  ip = 10.0.0.33 status = 0 setup_time = 1729131084208281 primary = 0 ],  ] ] ]
2024-10-17 Thu 02:11:52:780.999 UTC  cn-jenkins-deploy-platform-k8s-manifests-101-1 [Thread 140693219460864, Pid 809]: SANDESH: Send FAILED: 1729131112780769  [SYS_NOTICE]: VrouterStats: data= [ name = cn-jenkins-deploy-platform-k8s-manifests-101-1.cluster.local in_tpkts = 44253 in_bytes = 3178060 out_tpkts = 119689 out_bytes = 1251059528 exception_packets = 28 exception_packets_dropped = 10 raw_vhost_stats= [  name = vhost0 in_pkts = 44253 in_bytes = 3178060 out_pkts = 119689 out_bytes = 1251059528 speed = -1 duplexity = -1 ] raw_vhost_drop_stats= [  ds_discard = 0 ds_pull = 0 ds_invalid_if = 0 ds_invalid_arp = 0 ds_trap_no_if = 0 ds_nowhere_to_go = 0 ds_flow_queue_limit_exceeded = 0 ds_flow_no_memory = 0 ds_flow_invalid_protocol = 0 ds_flow_nat_no_rflow = 0 ds_flow_action_drop = 0 ds_flow_action_invalid = 0 ds_flow_unusable = 0 ds_flow_table_full = 0 ds_interface_tx_discard = 0 ds_interface_drop = 80 ds_duplicated = 0 ds_push = 0 ds_ttl_exceeded = 0 ds_invalid_nh = 1 ds_invalid_label = 0 ds_invalid_protocol = 0 ds_interface_rx_discard = 0 ds_invalid_mcast_source = 0 ds_head_alloc_fail = 0 ds_pcow_fail = 0 ds_mcast_clone_fail = 0 ds_rewrite_fail = 0 ds_misc = 0 ds_invalid_packet = 0 ds_cksum_err = 0 ds_no_fmd = 0 ds_invalid_vnid = 0 ds_frag_err = 0 ds_invalid_source = 0 ds_mcast_df_bit = 0 ds_l2_no_route = 0 ds_vlan_fwd_tx = 0 ds_vlan_fwd_enq = 0 ds_drop_new_flow = 0 ds_flow_evict = 0 ds_trap_original = 0 ds_no_memory = 0 ds_fragment_queue_fail = 0 ds_no_frag_entry = 0 ds_icmp_error = 0 ds_clone_fail = 0 ds_drop_pkts = 81 ] raw_drop_stats= [  ds_discard = 0 ds_pull = 0 ds_invalid_if = 0 ds_invalid_arp = 0 ds_trap_no_if = 0 ds_nowhere_to_go = 0 ds_flow_queue_limit_exceeded = 0 ds_flow_no_memory = 0 ds_flow_invalid_protocol = 0 ds_flow_nat_no_rflow = 0 ds_flow_action_drop = 0 ds_flow_action_invalid = 0 ds_flow_unusable = 0 ds_flow_table_full = 0 ds_interface_tx_discard = 0 ds_interface_drop = 80 ds_duplicated = 0 ds_push = 0 ds_ttl_exceeded = 0 ds_invalid_nh = 1 ds_invalid_label = 0 ds_invalid_protocol = 0 ds_interface_rx_discard = 0 ds_invalid_mcast_source = 0 ds_head_alloc_fail = 0 ds_pcow_fail = 0 ds_mcast_clone_fail = 0 ds_rewrite_fail = 0 ds_misc = 0 ds_invalid_packet = 0 ds_cksum_err = 0 ds_no_fmd = 0 ds_invalid_vnid = 0 ds_frag_err = 0 ds_invalid_source = 0 ds_mcast_df_bit = 0 ds_l2_no_route = 0 ds_vlan_fwd_tx = 0 ds_vlan_fwd_enq = 0 ds_drop_new_flow = 0 ds_flow_evict = 0 ds_trap_original = 0 ds_no_memory = 0 ds_fragment_queue_fail = 0 ds_no_frag_entry = 0 ds_icmp_error = 0 ds_clone_fail = 0 ds_drop_pkts = 81 ] total_in_bandwidth_utilization = 4.90571e-10 total_out_bandwidth_utilization = 1.22394e-12 flow_rate= [  added_flows = 0 max_flow_adds_per_second = 0 min_flow_adds_per_second = 0 deleted_flows = 0 max_flow_deletes_per_second = 0 min_flow_deletes_per_second = 0 active_flows = 0 hold_flows = 0 ] phy_band_in_bps= [  [ _iter260->first = eth0 _iter260->second = 90494338,  ] ] phy_band_out_bps= [  [ _iter261->first = eth0 _iter261->second = 225777,  ] ] raw_phy_if_stats= [  [ _iter262->first = eth0 [  in_pkts = 119634 in_bytes = 1251057218 out_pkts = 44254 out_bytes = 3178078 ],  ] ] raw_phy_if_drop_stats= [  [ _iter263->first = eth0 [  ds_discard = 0 ds_pull = 0 ds_invalid_if = 0 ds_invalid_arp = 0 ds_trap_no_if = 0 ds_nowhere_to_go = 0 ds_flow_queue_limit_exceeded = 0 ds_flow_no_memory = 0 ds_flow_invalid_protocol = 0 ds_flow_nat_no_rflow = 0 ds_flow_action_drop = 0 ds_flow_action_invalid = 0 ds_flow_unusable = 0 ds_flow_table_full = 0 ds_interface_tx_discard = 0 ds_interface_drop = 80 ds_duplicated = 0 ds_push = 0 ds_ttl_exceeded = 0 ds_invalid_nh = 0 ds_invalid_label = 0 ds_invalid_protocol = 0 ds_interface_rx_discard = 0 ds_invalid_mcast_source = 0 ds_head_alloc_fail = 0 ds_pcow_fail = 0 ds_mcast_clone_fail = 0 ds_rewrite_fail = 0 ds_misc = 0 ds_invalid_packet = 0 ds_cksum_err = 0 ds_no_fmd = 0 ds_invalid_vnid = 0 ds_frag_err = 0 ds_invalid_source = 0 ds_mcast_df_bit = 0 ds_l2_no_route = 0 ds_vlan_fwd_tx = 0 ds_vlan_fwd_enq = 0 ds_drop_new_flow = 0 ds_flow_evict = 0 ds_trap_original = 0 ds_no_memory = 0 ds_fragment_queue_fail = 0 ds_no_frag_entry = 0 ds_icmp_error = 0 ds_clone_fail = 0 ds_drop_pkts = 80 ],  ] ] ]
2024-10-17 Thu 02:11:52:781.053 UTC  cn-jenkins-deploy-platform-k8s-manifests-101-1 [Thread 140693219460864, Pid 809]: SANDESH: Send FAILED: 1729131112781026  [SYS_NOTICE]: VrouterControlStatsTrace: data= [ name = cn-jenkins-deploy-platform-k8s-manifests-101-1.cluster.local raw_xmpp_stats= [  [ _iter306->first = 10.0.0.33 [  reconnects = 1 in_msgs = 0 out_msgs = 1 ],  ] ] raw_ifmap_stats= [  [ _iter307->first = link_delete_parse_errors _iter307->second = 0,  _iter307->first = link_update_parse_errors _iter307->second = 0,  _iter307->first = node_delete_parse_errors _iter307->second = 0,  _iter307->first = node_update_parse_errors _iter307->second = 0,  ] ] raw_rt_table_size= [  [ _iter308->first = default-domain:default-project:ip-fabric:__default__ [  inet4_unicast = 7 inet4_multicast = 0 evpn = 0 bridge = 5 inet6_unicast = 2 ],  _iter308->first = default-domain:default-project:ip-fabric:ip-fabric [  inet4_unicast = 2 inet4_multicast = 0 evpn = 1 bridge = 2 inet6_unicast = 0 ],  ] ] ]
2024-10-17 Thu 02:11:55:351.751 UTC  cn-jenkins-deploy-platform-k8s-manifests-101-1 [Thread 140693219460864, Pid 809]: SANDESH: Send FAILED: 1729131115349716  [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-k8s-manifests-101-1.cluster.local process_status= [  [ [  module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [  [ [  type = XMPP name = control-node:10.0.0.33 server_addrs= [  [ (*_iter6) = 10.0.0.33:5269,  ] ] status = Down description = Established ],  [  type = XMPP name = dns-server:10.0.0.33 server_addrs= [  [ (*_iter6) = 10.0.0.33:53,  ] ] status = Down description = Connect ],  [  type = Collector name =  server_addrs= [  [ (*_iter6) = 10.0.0.33:8086,  ] ] status = Initializing description = Idle : EvIdleHoldTimerExpired -> Connect ],  ] ] flag_infos= [  [ ] ] description = XMPP:control-node:10.0.0.33, XMPP:dns-server:10.0.0.33, Collector connection down, No Configuration for self ],  ] ] ]
2024-10-17 Thu 02:11:55:351.959 UTC  cn-jenkins-deploy-platform-k8s-manifests-101-1 [Thread 140693219460864, Pid 809]: SANDESH: Send FAILED: 1729131115351706  [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-k8s-manifests-101-1.cluster.local:Compute:contrail-vrouter-agent:0 client_info= [  status = Connect successful_connections = 0 pid = 809 http_port = 8085 start_time = 1729131022563768 collector_name =  collector_ip = 10.0.0.33:8086 collector_list= [  [ (*_iter40) = 10.0.0.33:8086,  ] ] rx_socket_stats= [  bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration =  errors = 0 ] tx_socket_stats= [  bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration =  errors = 0 ] ] msg_type_agg= [  [ _iter58->first = ComputeCpuStateTrace [  messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ],  _iter58->first = NodeStatusUVE [  messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 21 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ],  _iter58->first = SandeshModuleClientTrace [  messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 22 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ],  _iter58->first = TcpServerMessageLog [  messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ],  _iter58->first = TcpSessionMessageLog [  messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 53 messages_sent_dropped_sending_to_syslog = 0 ],  _iter58->first = UveVirtualNetworkAgentTrace [  messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ],  _iter58->first = UveVrouterAgent [  messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 3 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ],  _iter58->first = VersionInfoLog [  messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ],  _iter58->first = VrfObjectLog [  messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ],  _iter58->first = VrouterControlStatsTrace [  messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 3 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ],  _iter58->first = VrouterStats [  messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 4 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ],  _iter58->first = XmppConnectionLog [  messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 54 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ],  _iter58->first = XmppEventLog [  messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 15 messages_sent_dropped_sending_to_syslog = 0 ],  ] ] tx_msg_agg= [  [ _iter59->first = dropped_client_send_failed _iter59->second = 0,  _iter59->first = dropped_header_write_failed _iter59->second = 0,  _iter59->first = dropped_no_client _iter59->second = 1,  _iter59->first = dropped_no_queue _iter59->second = 0,  _iter59->first = dropped_no_session _iter59->second = 0,  _iter59->first = dropped_queue_level _iter59->second = 0,  _iter59->first = dropped_rate_limited _iter59->second = 0,  _iter59->first = dropped_sending_disabled _iter59->second = 72,  _iter59->first = dropped_sending_to_syslog _iter59->second = 0,  _iter59->first = dropped_session_not_connected _iter59->second = 0,  _iter59->first = dropped_validation_failed _iter59->second = 0,  _iter59->first = dropped_write_failed _iter59->second = 0,  _iter59->first = dropped_wrong_client_sm_state _iter59->second = 111,  _iter59->first = sent _iter59->second = 0,  ] ] ]
2024-10-17 Thu 02:11:55:353.128 UTC  cn-jenkins-deploy-platform-k8s-manifests-101-1 [Thread 140693202667264, Pid 809]: SANDESH: Send FAILED: 1729131115352781  [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-k8s-manifests-101-1.cluster.local process_status= [  [ [  module_id = contrail-vrouter-agent instance_id = 0 state = Non-Functional connection_infos= [  [ [  type = XMPP name = control-node:10.0.0.33 server_addrs= [  [ (*_iter6) = 10.0.0.33:5269,  ] ] status = Down description = Established ],  [  type = XMPP name = dns-server:10.0.0.33 server_addrs= [  [ (*_iter6) = 10.0.0.33:53,  ] ] status = Down description = Connect ],  [  type = Collector name =  server_addrs= [  [ (*_iter6) = 10.0.0.33:8086,  ] ] status = Initializing description = Connect : EvTcpConnected ],  ] ] flag_infos= [  [ ] ] description = XMPP:control-node:10.0.0.33, XMPP:dns-server:10.0.0.33, Collector connection down, No Configuration for self ],  ] ] ]
2024-10-17 Thu 02:11:55:353.337 UTC  cn-jenkins-deploy-platform-k8s-manifests-101-1 [Thread 140693202667264, Pid 809]: SANDESH: Send FAILED: 1729131115353099  [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-k8s-manifests-101-1.cluster.local:Compute:contrail-vrouter-agent:0 client_info= [  status = ClientInit successful_connections = 1 pid = 809 http_port = 8085 start_time = 1729131022563768 collector_name =  collector_ip = 10.0.0.33:8086 collector_list= [  [ (*_iter40) = 10.0.0.33:8086,  ] ] rx_socket_stats= [  bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration =  errors = 0 ] tx_socket_stats= [  bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration =  errors = 0 ] ] msg_type_agg= [  [ _iter58->first = ComputeCpuStateTrace [  messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ],  _iter58->first = NodeStatusUVE [  messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 22 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ],  _iter58->first = SandeshModuleClientTrace [  messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 23 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ],  _iter58->first = TcpServerMessageLog [  messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ],  _iter58->first = TcpSessionMessageLog [  messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 54 messages_sent_dropped_sending_to_syslog = 0 ],  _iter58->first = UveVirtualNetworkAgentTrace [  messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ],  _iter58->first = UveVrouterAgent [  messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 3 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ],  _iter58->first = VersionInfoLog [  messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ],  _iter58->first = VrfObjectLog [  messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 2 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ],  _iter58->first = VrouterControlStatsTrace [  messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 3 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ],  _iter58->first = VrouterStats [  messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 4 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ],  _iter58->first = XmppConnectionLog [  messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 54 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ],  _iter58->first = XmppEventLog [  messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 15 messages_sent_dropped_sending_to_syslog = 0 ],  ] ] tx_msg_agg= [  [ _iter59->first = dropped_client_send_failed _iter59->second = 0,  _iter59->first = dropped_header_write_failed _iter59->second = 0,  _iter59->first = dropped_no_client _iter59->second = 1,  _iter59->first = dropped_no_queue _iter59->second = 0,  _iter59->first = dropped_no_session _iter59->second = 0,  _iter59->first = dropped_queue_level _iter59->second = 0,  _iter59->first = dropped_rate_limited _iter59->second = 0,  _iter59->first = dropped_sending_disabled _iter59->second = 73,  _iter59->first = dropped_sending_to_syslog _iter59->second = 0,  _iter59->first = dropped_session_not_connected _iter59->second = 0,  _iter59->first = dropped_validation_failed _iter59->second = 0,  _iter59->first = dropped_write_failed _iter59->second = 0,  _iter59->first = dropped_wrong_client_sm_state _iter59->second = 113,  _iter59->first = sent _iter59->second = 0,  ] ] ]
2024-10-17 Thu 02:11:55:880.445 UTC  cn-jenkins-deploy-platform-k8s-manifests-101-1 [Thread 140693376776512, Pid 809]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer cn-jenkins-deploy-platform-k8s-manifests-101-1.cluster.local:10.0.0.33 RECV  Mode Client:  Event: Tcp Connected  peer ip: 10.0.0.33 ( network-dns@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1295
2024-10-17 Thu 02:11:56:203.493 UTC  cn-jenkins-deploy-platform-k8s-manifests-101-1 [Thread 140693376776512, Pid 809]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer cn-jenkins-deploy-platform-k8s-manifests-101-1.cluster.local:10.0.0.33 RECV  Mode Client:  Event: Tcp Connected  peer ip: 10.0.0.33 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1295
2024-10-17 Thu 02:11:56:232.310 UTC  cn-jenkins-deploy-platform-k8s-manifests-101-1 [Thread 140693376776512, Pid 809]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer cn-jenkins-deploy-platform-k8s-manifests-101-1.cluster.local:10.0.0.33 RECV  Mode Client:  Event: Tcp Connection Closed  peer ip: 10.0.0.33 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1310
2024-10-17 Thu 02:12:23:504.982 UTC  cn-jenkins-deploy-platform-k8s-manifests-101-1 [Thread 140693376776512, Pid 809]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer cn-jenkins-deploy-platform-k8s-manifests-101-1.cluster.local:10.0.0.33 RECV  Mode Client:  Event: Tcp Connected  peer ip: 10.0.0.33 ( network-control@contrailsystems.com ) controller/src/xmpp/xmpp_state_machine.cc 1295
2024-10-17 Thu 02:15:35:277.225 UTC  cn-jenkins-deploy-platform-k8s-manifests-101-1 [Thread 140693223659264, Pid 809]: Error <19: No such device> querying mac-address for interface <tapeth0-82ebd4> Agent-index <8>
2024-10-17 Thu 02:15:41:922.382 UTC  cn-jenkins-deploy-platform-k8s-manifests-101-1 [Thread 140693215262464, Pid 809]: ID-PERM not set for object <default-domain:k8s-default:f9e3cbf5-370d-4045-ab7e-d865cf78c62b> Table <__ifmap__.service_instance.0>. Ignoring it
2024-10-17 Thu 02:15:42:751.234 UTC  cn-jenkins-deploy-platform-k8s-manifests-101-1 [Thread 140693223659264, Pid 809]: Error <19: No such device> querying mac-address for interface <tapeth0-412b77> Agent-index <6>
2024-10-17 Thu 02:15:42:754.352 UTC  cn-jenkins-deploy-platform-k8s-manifests-101-1 [Thread 140693223659264, Pid 809]: Error <19: No such device> querying mac-address for interface <tapeth0-412b77> Agent-index <6>
2024-10-17 Thu 02:15:47:844.717 UTC  cn-jenkins-deploy-platform-k8s-manifests-101-1 [Thread 140693215262464, Pid 809]: Error <19: No such device> querying mac-address for interface <tapeth0-a2de9f> Agent-index <5>
2024-10-17 Thu 02:19:36:118.873 UTC  cn-jenkins-deploy-platform-k8s-manifests-101-1 [Thread 140693227857664, Pid 809]: Error <19: No such device> querying mac-address for interface <tapeth0-f62362> Agent-index <5>
2024-10-17 Thu 02:19:36:140.178 UTC  cn-jenkins-deploy-platform-k8s-manifests-101-1 [Thread 140693215262464, Pid 809]: Error <19: No such device> querying mac-address for interface <tapeth0-00fe2b> Agent-index <7>
2024-10-17 Thu 02:19:36:173.723 UTC  cn-jenkins-deploy-platform-k8s-manifests-101-1 [Thread 140693223659264, Pid 809]: Error <19: No such device> querying mac-address for interface <tapeth0-e6ada8> Agent-index <6>
2024-10-17 Thu 02:19:36:178.701 UTC  cn-jenkins-deploy-platform-k8s-manifests-101-1 [Thread 140693223659264, Pid 809]: Error <19: No such device> querying mac-address for interface <tapeth0-e6ada8> Agent-index <6>
2024-10-17 Thu 02:19:36:188.003 UTC  cn-jenkins-deploy-platform-k8s-manifests-101-1 [Thread 140693223659264, Pid 809]: Error <19: No such device> querying mac-address for interface <tapeth0-e6ada8> Agent-index <6>
2024-10-17 Thu 02:19:36:210.018 UTC  cn-jenkins-deploy-platform-k8s-manifests-101-1 [Thread 140693223659264, Pid 809]: Error <19: No such device> querying mac-address for interface <tapeth0-a898f0> Agent-index <8>
2024-10-17 Thu 02:20:06:089.724 UTC  cn-jenkins-deploy-platform-k8s-manifests-101-1 [Thread 140693215262464, Pid 809]: Error <19: No such device> querying mac-address for interface <tapeth0-88b208> Agent-index <10>
2024-10-17 Thu 02:20:06:119.998 UTC  cn-jenkins-deploy-platform-k8s-manifests-101-1 [Thread 140693211064064, Pid 809]: Error <19: No such device> querying mac-address for interface <tapeth0-949ace> Agent-index <11>
2024-10-17 Thu 02:20:06:149.729 UTC  cn-jenkins-deploy-platform-k8s-manifests-101-1 [Thread 140693219460864, Pid 809]: Error <19: No such device> querying mac-address for interface <tapeth0-5645e7> Agent-index <12>
2024-10-17 Thu 02:20:06:170.053 UTC  cn-jenkins-deploy-platform-k8s-manifests-101-1 [Thread 140693219460864, Pid 809]: Error <19: No such device> querying mac-address for interface <tapeth0-db94b0> Agent-index <9>