2025-02-10 Mon 01:58:34:864.865 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604327573248, Pid 78556]: Wrong state: Connect for event: EvSandeshSend message: SandeshModuleClientTrace 2025-02-10 Mon 01:58:34:864.899 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604327573248, Pid 78556]: Processing scm::EvTcpConnected in state Connect 2025-02-10 Mon 01:58:34:864.908 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604327573248, Pid 78556]: Connect : EvTcpConnected : Cancelling Connect timer 2025-02-10 Mon 01:58:34:864.941 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604327573248, Pid 78556]: ClientInit 2025-02-10 Mon 01:58:34:864.962 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604327573248, Pid 78556]: Sending Ctrl Message for cn-jenkins-deploy-platform-ansible-os-2705-3:contrail-dns:0:Control count 2 2025-02-10 Mon 01:58:34:865.046 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604327573248, Pid 78556]: Processing scm::EvSandeshSend in state ClientInit 2025-02-10 Mon 01:58:34:865.055 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604327573248, Pid 78556]: ClientInit : EvSandeshSend : NodeStatusUVE 2025-02-10 Mon 01:58:34:865.077 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604327573248, Pid 78556]: SANDESH: Send FAILED: 1739152714864932 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-2705-3 process_status= [ [ [ module_id = contrail-dns instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.30:8086, ] ] status = Initializing description = Connect : EvTcpConnected ], [ type = Database name = Cassandra server_addrs= [ [ (*_iter6) = 10.0.0.17:9041, (*_iter6) = 10.0.0.12:9041, (*_iter6) = 10.0.0.30:9041, ] ] status = Up description = Established Cassandra connection ], [ type = Database name = RabbitMQ server_addrs= [ [ (*_iter6) = 10.0.0.17:5673, (*_iter6) = 10.0.0.12:5673, (*_iter6) = 10.0.0.30:5673, ] ] status = Up description = RabbitMQ connection established ], ] ] flag_infos= [ [ ] ] description = Collector connection down ], ] ] ] 2025-02-10 Mon 01:58:34:865.084 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604327573248, Pid 78556]: Received UVE message in wrong state : NodeStatusUVE 2025-02-10 Mon 01:58:34:865.092 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604327573248, Pid 78556]: Processing scm::EvSandeshSend in state ClientInit 2025-02-10 Mon 01:58:34:865.099 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604327573248, Pid 78556]: ClientInit : EvSandeshSend : SandeshCtrlClientToServer 2025-02-10 Mon 01:58:34:865.109 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604327573248, Pid 78556]: Processing scm::EvSandeshSend in state ClientInit 2025-02-10 Mon 01:58:34:865.116 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604327573248, Pid 78556]: ClientInit : EvSandeshSend : SandeshModuleClientTrace 2025-02-10 Mon 01:58:34:865.327 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604327573248, Pid 78556]: SANDESH: Send FAILED: 1739152714865035 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-2705-3:Control:contrail-dns:0 client_info= [ status = ClientInit successful_connections = 2 pid = 78556 http_port = 8092 start_time = 1739152566176088 collector_name = collector_ip = 10.0.0.30:8086 collector_list= [ [ (*_iter40) = 10.0.0.17:8086, (*_iter40) = 10.0.0.30:8086, (*_iter40) = 10.0.0.12:8086, ] ] rx_socket_stats= [ bytes = 1017 calls = 2 average_bytes = 508 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 144901 calls = 11 average_bytes = 13172 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ConfigClientGetRowError [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 6 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigClientMgrDebug [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 647 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigurationMalformedReferenceWarningLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 17 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = DnsOperational [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 14 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapLinkOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 469 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapNodeOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 271 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 3 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 11 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlClientToServer [ messages_sent = 1 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlServerToClient [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 8 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 8 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 8 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveDnsInfo [ messages_sent = 1 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 1434, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 19, _iter59->first = sent _iter59->second = 13, ] ] ] 2025-02-10 Mon 01:58:34:865.350 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604327573248, Pid 78556]: Received UVE message in wrong state : SandeshModuleClientTrace 2025-02-10 Mon 01:58:35:003.208 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604323374848, Pid 78556]: OnMessage control in state: ClientInit 2025-02-10 Mon 01:58:35:003.255 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604323374848, Pid 78556]: Processing scm::EvSandeshMessageRecv in state ClientInit 2025-02-10 Mon 01:58:35:003.265 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604323374848, Pid 78556]: ClientInit : EvSandeshMessageRecv 2025-02-10 Mon 01:58:35:003.289 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604323374848, Pid 78556]: Received Ctrl Message with size 0 2025-02-10 Mon 01:58:35:003.309 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604323374848, Pid 78556]: SyncUVE Syncing ObjectDns val name = cn-jenkins-deploy-platform-ansible-os-2705-3 start_time = 1739152566954394 build_info = {"build-info":[{"build-version":"nightly.r24.1.centos","build-time":"2025-02-06 00:13:12.562772","build-user":"root","build-hostname":"jenkins-build-prepare-sandbox-7691-1.novalocal","build-id":"nightly.r24.1.centos-nightly_r24.1_centos.el7@contrail","build-number":"unknown"}]} collector = 10.0.0.17:8086 self_ip_list= [ [ (*_iter12) = 10.20.0.8, ] ] proxy seq 1 2025-02-10 Mon 01:58:35:003.323 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604323374848, Pid 78556]: SyncAllMaps for DnsState:0 period 0 cycle 0 without seqno , total = 1 2025-02-10 Mon 01:58:35:003.606 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604323374848, Pid 78556]: SyncUVE Syncing ObjectGeneratorInfo val name = cn-jenkins-deploy-platform-ansible-os-2705-3:Control:contrail-dns:0 client_info= [ status = ClientInit successful_connections = 2 pid = 78556 http_port = 8092 start_time = 1739152566176088 collector_name = collector_ip = 10.0.0.30:8086 collector_list= [ [ (*_iter40) = 10.0.0.17:8086, (*_iter40) = 10.0.0.30:8086, (*_iter40) = 10.0.0.12:8086, ] ] rx_socket_stats= [ bytes = 1017 calls = 2 average_bytes = 508 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 144901 calls = 11 average_bytes = 13172 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] sm_queue_count = 0 sm_stats= [ ev_stats= [ [ [ event = scm::EvCollectorUpdate enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvSandeshMessageRecv enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvSandeshSend enqueues = 27 dequeues = 27 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvStart enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpConnected enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = enqueues = 31 dequeues = 31 enqueue_fails = 0 dequeue_fails = 0 ], ] ] state = Established state_since = 1739152600299116 last_state = Established last_event = scm::EvSandeshSend last_event_at = 1739152660286297 ] session_stats= [ num_recv_msg = 0 num_recv_msg_fail = 0 num_recv_fail = 0 num_send_msg = 11 num_send_msg_fail = 0 num_send_buffer_fail = 0 num_wait_msgq_enqueue = 0 num_wait_msgq_dequeue = 0 num_write_ready_cb_error = 0 ] session_rx_socket_stats= [ bytes = 1017 calls = 2 average_bytes = 508 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] session_tx_socket_stats= [ bytes = 115679 calls = 9 average_bytes = 12853 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] msg_type_agg= [ [ _iter58->first = ConfigClientGetRowError [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 6 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigClientMgrDebug [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 647 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigurationMalformedReferenceWarningLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 17 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = DnsOperational [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 14 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapLinkOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 469 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapNodeOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 271 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 3 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 11 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlClientToServer [ messages_sent = 1 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlServerToClient [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 8 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 8 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 8 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveDnsInfo [ messages_sent = 1 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 1434, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 19, _iter59->first = sent _iter59->second = 13, ] ] max_sm_queue_count = 5 proxy seq 17 2025-02-10 Mon 01:58:35:003.644 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604323374848, Pid 78556]: SyncAllMaps for ModuleClientState:0 period 0 cycle 0 without seqno , total = 1 2025-02-10 Mon 01:58:35:003.670 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604323374848, Pid 78556]: SyncUVE Syncing ObjectDns val name = cn-jenkins-deploy-platform-ansible-os-2705-3 process_status= [ [ [ module_id = contrail-dns instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.30:8086, ] ] status = Initializing description = Connect : EvTcpConnected ], [ type = Database name = Cassandra server_addrs= [ [ (*_iter6) = 10.0.0.17:9041, (*_iter6) = 10.0.0.12:9041, (*_iter6) = 10.0.0.30:9041, ] ] status = Up description = Established Cassandra connection ], [ type = Database name = RabbitMQ server_addrs= [ [ (*_iter6) = 10.0.0.17:5673, (*_iter6) = 10.0.0.12:5673, (*_iter6) = 10.0.0.30:5673, ] ] status = Up description = RabbitMQ connection established ], ] ] flag_infos= [ [ ] ] description = Collector connection down ], ] ] proxy seq 14 2025-02-10 Mon 01:58:35:003.683 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604323374848, Pid 78556]: SyncAllMaps for NodeStatus:0 period 0 cycle 0 without seqno , total = 1 2025-02-10 Mon 01:58:35:003.705 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604323374848, Pid 78556]: SyncAllMaps for PeerFlapData:0 period 0 cycle 0 without seqno , total = 0 2025-02-10 Mon 01:58:35:003.713 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604323374848, Pid 78556]: SyncAllMaps for PeerStatsData:0 period 0 cycle 0 without seqno , total = 0 2025-02-10 Mon 01:58:35:003.721 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604323374848, Pid 78556]: SyncAllMaps for XmppPeerInfoData:0 period 0 cycle 0 without seqno , total = 0 2025-02-10 Mon 01:58:35:003.755 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604323374848, Pid 78556]: Established 2025-02-10 Mon 01:58:35:066.530 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604735187072, Pid 78556]: 10.0.0.30:58428::10.0.0.30:8086(-1) OnSessionEvent TCP Connection Closed 2025-02-10 Mon 01:58:35:066.559 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604735187072, Pid 78556]: 10.0.0.30:58428::10.0.0.30:8086(-1):EvTcpClose 2025-02-10 Mon 01:58:35:066.612 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604562466560, Pid 78556]: Processing scm::EvTcpClose in state Established 2025-02-10 Mon 01:58:35:066.625 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604562466560, Pid 78556]: Established : EvTcpClose 2025-02-10 Mon 01:58:35:066.898 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604562466560, Pid 78556]: Connect : Start Connect timer 10.0.0.12:8086 2025-02-10 Mon 01:58:35:066.996 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604562466560, Pid 78556]: Processing scm::EvSandeshSend in state Connect 2025-02-10 Mon 01:58:35:067.232 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604562466560, Pid 78556]: SANDESH: Send FAILED: 1739152715066753 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-2705-3:Control:contrail-dns:0 client_info= [ status = Established successful_connections = 2 pid = 78556 http_port = 8092 start_time = 1739152566176088 collector_name = cn-jenkins-deploy-platform-ansible-os-2705-3. collector_ip = 10.0.0.12:8086 collector_list= [ [ (*_iter40) = 10.0.0.17:8086, (*_iter40) = 10.0.0.30:8086, (*_iter40) = 10.0.0.12:8086, ] ] rx_socket_stats= [ bytes = 2034 calls = 4 average_bytes = 508 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 208607 calls = 16 average_bytes = 13037 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ConfigClientGetRowError [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 6 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigClientMgrDebug [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 647 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigurationMalformedReferenceWarningLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 17 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = DnsOperational [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 14 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapLinkOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 469 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapNodeOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 271 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 6 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 12 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlClientToServer [ messages_sent = 2 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlServerToClient [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 10 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 9 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 8 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveDnsInfo [ messages_sent = 2 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 1434, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 21, _iter59->first = sent _iter59->second = 20, ] ] ] 2025-02-10 Mon 01:58:35:067.257 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604562466560, Pid 78556]: Wrong state: Connect for event: EvSandeshSend message: SandeshModuleClientTrace 2025-02-10 Mon 01:58:35:067.268 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604562466560, Pid 78556]: Processing scm::EvSandeshSend in state Connect 2025-02-10 Mon 01:58:35:067.292 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604562466560, Pid 78556]: SANDESH: Send FAILED: 1739152715066789 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-2705-3 process_status= [ [ [ module_id = contrail-dns instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.30:8086, ] ] status = Initializing description = Established : EvTcpClose -> Connect ], [ type = Database name = Cassandra server_addrs= [ [ (*_iter6) = 10.0.0.17:9041, (*_iter6) = 10.0.0.12:9041, (*_iter6) = 10.0.0.30:9041, ] ] status = Up description = Established Cassandra connection ], [ type = Database name = RabbitMQ server_addrs= [ [ (*_iter6) = 10.0.0.17:5673, (*_iter6) = 10.0.0.12:5673, (*_iter6) = 10.0.0.30:5673, ] ] status = Up description = RabbitMQ connection established ], ] ] flag_infos= [ [ ] ] description = Collector connection down ], ] ] ] 2025-02-10 Mon 01:58:35:067.299 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604562466560, Pid 78556]: Wrong state: Connect for event: EvSandeshSend message: NodeStatusUVE 2025-02-10 Mon 01:58:35:067.308 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604562466560, Pid 78556]: Processing scm::EvTcpDeleteSession in state Connect 2025-02-10 Mon 01:58:35:067.335 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604562466560, Pid 78556]: Processing scm::EvSandeshSend in state Connect 2025-02-10 Mon 01:58:35:067.529 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604562466560, Pid 78556]: SANDESH: Send FAILED: 1739152715066971 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-2705-3:Control:contrail-dns:0 client_info= [ status = Connect successful_connections = 2 pid = 78556 http_port = 8092 start_time = 1739152566176088 collector_name = collector_ip = 10.0.0.12:8086 collector_list= [ [ (*_iter40) = 10.0.0.17:8086, (*_iter40) = 10.0.0.30:8086, (*_iter40) = 10.0.0.12:8086, ] ] rx_socket_stats= [ bytes = 2034 calls = 4 average_bytes = 508 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 208607 calls = 16 average_bytes = 13037 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ConfigClientGetRowError [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 6 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigClientMgrDebug [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 647 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigurationMalformedReferenceWarningLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 17 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = DnsOperational [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 14 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapLinkOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 469 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapNodeOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 271 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 6 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 12 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlClientToServer [ messages_sent = 2 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlServerToClient [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 10 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 9 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 8 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveDnsInfo [ messages_sent = 2 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 1434, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 21, _iter59->first = sent _iter59->second = 20, ] ] ] 2025-02-10 Mon 01:58:35:067.550 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604562466560, Pid 78556]: Wrong state: Connect for event: EvSandeshSend message: SandeshModuleClientTrace 2025-02-10 Mon 01:58:35:072.852 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604735187072, Pid 78556]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.0.0.30:38160::10.0.0.12:8086(-1) < Active session connection complete src/contrail-common/io/tcp_session.cc 280 2025-02-10 Mon 01:58:35:072.878 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604735187072, Pid 78556]: 10.0.0.30:38160::10.0.0.12:8086(-1) OnSessionEvent TCP Connected 2025-02-10 Mon 01:58:35:072.886 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604735187072, Pid 78556]: 10.0.0.30:38160::10.0.0.12:8086(-1):EvTcpConnected 2025-02-10 Mon 01:58:35:072.940 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604331771648, Pid 78556]: Processing scm::EvTcpConnected in state Connect 2025-02-10 Mon 01:58:35:072.968 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604331771648, Pid 78556]: Connect : EvTcpConnected : Cancelling Connect timer 2025-02-10 Mon 01:58:35:073.013 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604331771648, Pid 78556]: ClientInit 2025-02-10 Mon 01:58:35:073.029 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604331771648, Pid 78556]: Sending Ctrl Message for cn-jenkins-deploy-platform-ansible-os-2705-3:contrail-dns:0:Control count 3 2025-02-10 Mon 01:58:35:073.141 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604331771648, Pid 78556]: Processing scm::EvSandeshSend in state ClientInit 2025-02-10 Mon 01:58:35:073.173 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604331771648, Pid 78556]: ClientInit : EvSandeshSend : NodeStatusUVE 2025-02-10 Mon 01:58:35:073.201 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604331771648, Pid 78556]: SANDESH: Send FAILED: 1739152715073003 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-2705-3 process_status= [ [ [ module_id = contrail-dns instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.12:8086, ] ] status = Initializing description = Connect : EvTcpConnected ], [ type = Database name = Cassandra server_addrs= [ [ (*_iter6) = 10.0.0.17:9041, (*_iter6) = 10.0.0.12:9041, (*_iter6) = 10.0.0.30:9041, ] ] status = Up description = Established Cassandra connection ], [ type = Database name = RabbitMQ server_addrs= [ [ (*_iter6) = 10.0.0.17:5673, (*_iter6) = 10.0.0.12:5673, (*_iter6) = 10.0.0.30:5673, ] ] status = Up description = RabbitMQ connection established ], ] ] flag_infos= [ [ ] ] description = Collector connection down ], ] ] ] 2025-02-10 Mon 01:58:35:073.209 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604331771648, Pid 78556]: Received UVE message in wrong state : NodeStatusUVE 2025-02-10 Mon 01:58:35:073.218 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604331771648, Pid 78556]: Processing scm::EvSandeshSend in state ClientInit 2025-02-10 Mon 01:58:35:073.225 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604331771648, Pid 78556]: ClientInit : EvSandeshSend : SandeshCtrlClientToServer 2025-02-10 Mon 01:58:35:073.237 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604331771648, Pid 78556]: Processing scm::EvSandeshSend in state ClientInit 2025-02-10 Mon 01:58:35:073.243 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604331771648, Pid 78556]: ClientInit : EvSandeshSend : SandeshModuleClientTrace 2025-02-10 Mon 01:58:35:073.450 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604331771648, Pid 78556]: SANDESH: Send FAILED: 1739152715073128 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-2705-3:Control:contrail-dns:0 client_info= [ status = ClientInit successful_connections = 3 pid = 78556 http_port = 8092 start_time = 1739152566176088 collector_name = collector_ip = 10.0.0.12:8086 collector_list= [ [ (*_iter40) = 10.0.0.17:8086, (*_iter40) = 10.0.0.30:8086, (*_iter40) = 10.0.0.12:8086, ] ] rx_socket_stats= [ bytes = 2034 calls = 4 average_bytes = 508 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 208607 calls = 16 average_bytes = 13037 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ConfigClientGetRowError [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 6 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigClientMgrDebug [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 647 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigurationMalformedReferenceWarningLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 17 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = DnsOperational [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 14 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapLinkOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 469 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapNodeOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 271 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 6 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 13 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlClientToServer [ messages_sent = 2 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlServerToClient [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 10 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 11 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 9 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveDnsInfo [ messages_sent = 2 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 1435, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 24, _iter59->first = sent _iter59->second = 20, ] ] ] 2025-02-10 Mon 01:58:35:073.468 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604331771648, Pid 78556]: Received UVE message in wrong state : SandeshModuleClientTrace 2025-02-10 Mon 01:58:35:110.930 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604735187072, Pid 78556]: 10.0.0.30:38160::10.0.0.12:8086(-1) OnSessionEvent TCP Connection Closed 2025-02-10 Mon 01:58:35:110.966 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604735187072, Pid 78556]: 10.0.0.30:38160::10.0.0.12:8086(-1):EvTcpClose 2025-02-10 Mon 01:58:35:111.029 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604562466560, Pid 78556]: Processing scm::EvTcpClose in state ClientInit 2025-02-10 Mon 01:58:35:111.061 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604562466560, Pid 78556]: ClientInit : EvTcpClose 2025-02-10 Mon 01:58:35:111.200 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604562466560, Pid 78556]: Return to idle with 5000 2025-02-10 Mon 01:58:35:111.210 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604562466560, Pid 78556]: Idle 2025-02-10 Mon 01:58:35:111.268 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604562466560, Pid 78556]: Processing scm::EvSandeshSend in state Idle 2025-02-10 Mon 01:58:35:111.467 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604562466560, Pid 78556]: SANDESH: Send FAILED: 1739152715111157 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-2705-3:Control:contrail-dns:0 client_info= [ status = ClientInit successful_connections = 3 pid = 78556 http_port = 8092 start_time = 1739152566176088 collector_name = collector_ip = 10.0.0.17:8086 collector_list= [ [ (*_iter40) = 10.0.0.17:8086, (*_iter40) = 10.0.0.30:8086, (*_iter40) = 10.0.0.12:8086, ] ] rx_socket_stats= [ bytes = 2034 calls = 4 average_bytes = 508 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 210217 calls = 17 average_bytes = 12365 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ConfigClientGetRowError [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 6 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigClientMgrDebug [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 647 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigurationMalformedReferenceWarningLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 17 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = DnsOperational [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 14 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapLinkOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 469 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapNodeOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 271 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 6 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 14 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlClientToServer [ messages_sent = 3 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlServerToClient [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 10 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 12 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 9 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveDnsInfo [ messages_sent = 2 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 1435, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 26, _iter59->first = sent _iter59->second = 21, ] ] ] 2025-02-10 Mon 01:58:35:111.494 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604562466560, Pid 78556]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-02-10 Mon 01:58:35:111.505 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604562466560, Pid 78556]: Processing scm::EvSandeshSend in state Idle 2025-02-10 Mon 01:58:35:111.528 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604562466560, Pid 78556]: SANDESH: Send FAILED: 1739152715111189 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-2705-3 process_status= [ [ [ module_id = contrail-dns instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.12:8086, ] ] status = Down description = ClientInit : EvTcpClose ], [ type = Database name = Cassandra server_addrs= [ [ (*_iter6) = 10.0.0.17:9041, (*_iter6) = 10.0.0.12:9041, (*_iter6) = 10.0.0.30:9041, ] ] status = Up description = Established Cassandra connection ], [ type = Database name = RabbitMQ server_addrs= [ [ (*_iter6) = 10.0.0.17:5673, (*_iter6) = 10.0.0.12:5673, (*_iter6) = 10.0.0.30:5673, ] ] status = Up description = RabbitMQ connection established ], ] ] flag_infos= [ [ ] ] description = Collector connection down ], ] ] ] 2025-02-10 Mon 01:58:35:111.535 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604562466560, Pid 78556]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-02-10 Mon 01:58:35:111.543 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604562466560, Pid 78556]: Processing scm::EvTcpDeleteSession in state Idle 2025-02-10 Mon 01:58:35:111.567 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604562466560, Pid 78556]: Processing scm::EvSandeshSend in state Idle 2025-02-10 Mon 01:58:35:111.768 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604562466560, Pid 78556]: SANDESH: Send FAILED: 1739152715111258 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-2705-3:Control:contrail-dns:0 client_info= [ status = Idle successful_connections = 3 pid = 78556 http_port = 8092 start_time = 1739152566176088 collector_name = collector_ip = 10.0.0.17:8086 collector_list= [ [ (*_iter40) = 10.0.0.17:8086, (*_iter40) = 10.0.0.30:8086, (*_iter40) = 10.0.0.12:8086, ] ] rx_socket_stats= [ bytes = 2034 calls = 4 average_bytes = 508 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 210217 calls = 17 average_bytes = 12365 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ConfigClientGetRowError [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 6 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigClientMgrDebug [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 647 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigurationMalformedReferenceWarningLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 17 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = DnsOperational [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 14 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapLinkOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 469 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapNodeOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 271 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 6 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 14 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlClientToServer [ messages_sent = 3 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlServerToClient [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 10 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 12 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 9 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveDnsInfo [ messages_sent = 2 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 1435, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 26, _iter59->first = sent _iter59->second = 21, ] ] ] 2025-02-10 Mon 01:58:35:111.784 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604562466560, Pid 78556]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-02-10 Mon 01:58:40:111.325 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604331771648, Pid 78556]: Processing scm::EvIdleHoldTimerExpired in state Idle 2025-02-10 Mon 01:58:40:111.594 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604331771648, Pid 78556]: Connect : Start Connect timer 10.0.0.17:8086 2025-02-10 Mon 01:58:40:111.741 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604331771648, Pid 78556]: Processing scm::EvSandeshSend in state Connect 2025-02-10 Mon 01:58:40:111.774 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604331771648, Pid 78556]: SANDESH: Send FAILED: 1739152720111410 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-2705-3 process_status= [ [ [ module_id = contrail-dns instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.17:8086, ] ] status = Initializing description = Idle : EvIdleHoldTimerExpired -> Connect ], [ type = Database name = Cassandra server_addrs= [ [ (*_iter6) = 10.0.0.17:9041, (*_iter6) = 10.0.0.12:9041, (*_iter6) = 10.0.0.30:9041, ] ] status = Up description = Established Cassandra connection ], [ type = Database name = RabbitMQ server_addrs= [ [ (*_iter6) = 10.0.0.17:5673, (*_iter6) = 10.0.0.12:5673, (*_iter6) = 10.0.0.30:5673, ] ] status = Up description = RabbitMQ connection established ], ] ] flag_infos= [ [ ] ] description = Collector connection down ], ] ] ] 2025-02-10 Mon 01:58:40:111.783 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604331771648, Pid 78556]: Wrong state: Connect for event: EvSandeshSend message: NodeStatusUVE 2025-02-10 Mon 01:58:40:111.792 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604331771648, Pid 78556]: Processing scm::EvSandeshSend in state Connect 2025-02-10 Mon 01:58:40:112.009 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604331771648, Pid 78556]: SANDESH: Send FAILED: 1739152720111723 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-2705-3:Control:contrail-dns:0 client_info= [ status = Connect successful_connections = 3 pid = 78556 http_port = 8092 start_time = 1739152566176088 collector_name = collector_ip = 10.0.0.17:8086 collector_list= [ [ (*_iter40) = 10.0.0.17:8086, (*_iter40) = 10.0.0.30:8086, (*_iter40) = 10.0.0.12:8086, ] ] rx_socket_stats= [ bytes = 2034 calls = 4 average_bytes = 508 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 210217 calls = 17 average_bytes = 12365 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ConfigClientGetRowError [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 6 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigClientMgrDebug [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 647 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigurationMalformedReferenceWarningLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 17 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = DnsOperational [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 14 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapLinkOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 469 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapNodeOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 271 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 6 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 15 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlClientToServer [ messages_sent = 3 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlServerToClient [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 10 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 14 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 9 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveDnsInfo [ messages_sent = 2 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 1435, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 29, _iter59->first = sent _iter59->second = 21, ] ] ] 2025-02-10 Mon 01:58:40:112.028 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604331771648, Pid 78556]: Wrong state: Connect for event: EvSandeshSend message: SandeshModuleClientTrace 2025-02-10 Mon 01:58:40:113.731 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604735187072, Pid 78556]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.0.0.30:56924::10.0.0.17:8086(-1) < Active session connection complete src/contrail-common/io/tcp_session.cc 280 2025-02-10 Mon 01:58:40:113.760 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604735187072, Pid 78556]: 10.0.0.30:56924::10.0.0.17:8086(-1) OnSessionEvent TCP Connected 2025-02-10 Mon 01:58:40:113.769 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604735187072, Pid 78556]: 10.0.0.30:56924::10.0.0.17:8086(-1):EvTcpConnected 2025-02-10 Mon 01:58:40:113.815 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604344366848, Pid 78556]: Processing scm::EvTcpConnected in state Connect 2025-02-10 Mon 01:58:40:113.841 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604344366848, Pid 78556]: Connect : EvTcpConnected : Cancelling Connect timer 2025-02-10 Mon 01:58:40:113.888 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604344366848, Pid 78556]: ClientInit 2025-02-10 Mon 01:58:40:113.905 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604344366848, Pid 78556]: Sending Ctrl Message for cn-jenkins-deploy-platform-ansible-os-2705-3:contrail-dns:0:Control count 4 2025-02-10 Mon 01:58:40:114.026 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604344366848, Pid 78556]: Processing scm::EvSandeshSend in state ClientInit 2025-02-10 Mon 01:58:40:114.036 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604344366848, Pid 78556]: ClientInit : EvSandeshSend : NodeStatusUVE 2025-02-10 Mon 01:58:40:114.061 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604344366848, Pid 78556]: SANDESH: Send FAILED: 1739152720113880 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-2705-3 process_status= [ [ [ module_id = contrail-dns instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.17:8086, ] ] status = Initializing description = Connect : EvTcpConnected ], [ type = Database name = Cassandra server_addrs= [ [ (*_iter6) = 10.0.0.17:9041, (*_iter6) = 10.0.0.12:9041, (*_iter6) = 10.0.0.30:9041, ] ] status = Up description = Established Cassandra connection ], [ type = Database name = RabbitMQ server_addrs= [ [ (*_iter6) = 10.0.0.17:5673, (*_iter6) = 10.0.0.12:5673, (*_iter6) = 10.0.0.30:5673, ] ] status = Up description = RabbitMQ connection established ], ] ] flag_infos= [ [ ] ] description = Collector connection down ], ] ] ] 2025-02-10 Mon 01:58:40:114.069 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604344366848, Pid 78556]: Received UVE message in wrong state : NodeStatusUVE 2025-02-10 Mon 01:58:40:114.077 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604344366848, Pid 78556]: Processing scm::EvSandeshSend in state ClientInit 2025-02-10 Mon 01:58:40:114.083 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604344366848, Pid 78556]: ClientInit : EvSandeshSend : SandeshCtrlClientToServer 2025-02-10 Mon 01:58:40:114.095 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604344366848, Pid 78556]: Processing scm::EvSandeshSend in state ClientInit 2025-02-10 Mon 01:58:40:114.101 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604344366848, Pid 78556]: ClientInit : EvSandeshSend : SandeshModuleClientTrace 2025-02-10 Mon 01:58:40:114.308 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604344366848, Pid 78556]: SANDESH: Send FAILED: 1739152720114013 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-2705-3:Control:contrail-dns:0 client_info= [ status = ClientInit successful_connections = 4 pid = 78556 http_port = 8092 start_time = 1739152566176088 collector_name = collector_ip = 10.0.0.17:8086 collector_list= [ [ (*_iter40) = 10.0.0.17:8086, (*_iter40) = 10.0.0.30:8086, (*_iter40) = 10.0.0.12:8086, ] ] rx_socket_stats= [ bytes = 2034 calls = 4 average_bytes = 508 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 210217 calls = 17 average_bytes = 12365 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ConfigClientGetRowError [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 6 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigClientMgrDebug [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 647 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigurationMalformedReferenceWarningLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 17 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = DnsOperational [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 14 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapLinkOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 469 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapNodeOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 271 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 6 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 16 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlClientToServer [ messages_sent = 3 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlServerToClient [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 10 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 15 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 10 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveDnsInfo [ messages_sent = 2 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 1436, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 31, _iter59->first = sent _iter59->second = 21, ] ] ] 2025-02-10 Mon 01:58:40:114.330 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604344366848, Pid 78556]: Received UVE message in wrong state : SandeshModuleClientTrace 2025-02-10 Mon 01:58:40:286.840 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604575061760, Pid 78556]: Processing scm::EvSandeshSend in state ClientInit 2025-02-10 Mon 01:58:40:286.882 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604575061760, Pid 78556]: ClientInit : EvSandeshSend : SandeshModuleClientTrace 2025-02-10 Mon 01:58:40:286.970 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604575061760, Pid 78556]: SANDESH: Send FAILED: 1739152720286677 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-2705-3:Control:contrail-dns:0 sm_queue_count = 0 sm_stats= [ ev_stats= [ [ [ event = scm::EvCollectorUpdate enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvIdleHoldTimerExpired enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvSandeshMessageRecv enqueues = 2 dequeues = 2 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvSandeshSend enqueues = 55 dequeues = 55 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvStart enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpClose enqueues = 3 dequeues = 3 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpConnected enqueues = 4 dequeues = 4 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpDeleteSession enqueues = 3 dequeues = 3 enqueue_fails = 0 dequeue_fails = 0 ], [ event = enqueues = 70 dequeues = 70 enqueue_fails = 0 dequeue_fails = 0 ], ] ] state = ClientInit state_since = 1739152720113887 last_state = ClientInit last_event = scm::EvSandeshSend last_event_at = 1739152720114094 ] session_stats= [ num_recv_msg = 0 num_recv_msg_fail = 0 num_recv_fail = 0 num_send_msg = 1 num_send_msg_fail = 0 num_send_buffer_fail = 0 num_wait_msgq_enqueue = 0 num_wait_msgq_dequeue = 0 num_write_ready_cb_error = 0 ] session_rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] session_tx_socket_stats= [ bytes = 1610 calls = 1 average_bytes = 1610 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] max_sm_queue_count = 6 ] 2025-02-10 Mon 01:58:40:286.982 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604575061760, Pid 78556]: Received UVE message in wrong state : SandeshModuleClientTrace 2025-02-10 Mon 01:58:40:286.991 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604575061760, Pid 78556]: Processing scm::EvSandeshSend in state ClientInit 2025-02-10 Mon 01:58:40:286.998 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604575061760, Pid 78556]: ClientInit : EvSandeshSend : SandeshModuleClientTrace 2025-02-10 Mon 01:58:40:287.185 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604575061760, Pid 78556]: SANDESH: Send FAILED: 1739152720286802 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-2705-3:Control:contrail-dns:0 client_info= [ status = ClientInit successful_connections = 4 pid = 78556 http_port = 8092 start_time = 1739152566176088 collector_name = collector_ip = 10.0.0.17:8086 collector_list= [ [ (*_iter40) = 10.0.0.17:8086, (*_iter40) = 10.0.0.30:8086, (*_iter40) = 10.0.0.12:8086, ] ] rx_socket_stats= [ bytes = 2034 calls = 4 average_bytes = 508 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 211827 calls = 18 average_bytes = 11768 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ConfigClientGetRowError [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 6 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigClientMgrDebug [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 647 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigurationMalformedReferenceWarningLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 17 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = DnsOperational [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 14 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapLinkOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 469 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapNodeOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 271 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 6 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 17 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlClientToServer [ messages_sent = 4 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlServerToClient [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 10 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 16 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 10 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveDnsInfo [ messages_sent = 2 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 1436, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 33, _iter59->first = sent _iter59->second = 22, ] ] ] 2025-02-10 Mon 01:58:40:287.212 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604575061760, Pid 78556]: Received UVE message in wrong state : SandeshModuleClientTrace 2025-02-10 Mon 01:58:42:113.995 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604323374848, Pid 78556]: 10.0.0.17:8086 EvConnectTimerExpired in state ClientInit 2025-02-10 Mon 01:58:42:114.070 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604554069760, Pid 78556]: Processing scm::EvConnectTimerExpired in state ClientInit 2025-02-10 Mon 01:58:42:114.100 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604554069760, Pid 78556]: ClientInit : EvConnectTimerExpired 2025-02-10 Mon 01:58:42:114.294 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604554069760, Pid 78556]: Return to idle with 5000 2025-02-10 Mon 01:58:42:114.305 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604554069760, Pid 78556]: Idle 2025-02-10 Mon 01:58:42:114.371 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604554069760, Pid 78556]: Processing scm::EvSandeshSend in state Idle 2025-02-10 Mon 01:58:42:114.571 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604554069760, Pid 78556]: SANDESH: Send FAILED: 1739152722114192 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-2705-3:Control:contrail-dns:0 client_info= [ status = ClientInit successful_connections = 4 pid = 78556 http_port = 8092 start_time = 1739152566176088 collector_name = collector_ip = 10.0.0.30:8086 collector_list= [ [ (*_iter40) = 10.0.0.17:8086, (*_iter40) = 10.0.0.30:8086, (*_iter40) = 10.0.0.12:8086, ] ] rx_socket_stats= [ bytes = 2034 calls = 4 average_bytes = 508 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 211827 calls = 18 average_bytes = 11768 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ConfigClientGetRowError [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 6 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigClientMgrDebug [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 647 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigurationMalformedReferenceWarningLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 17 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = DnsOperational [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 14 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapLinkOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 469 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapNodeOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 271 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 6 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 17 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlClientToServer [ messages_sent = 4 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlServerToClient [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 10 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 18 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 10 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveDnsInfo [ messages_sent = 2 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 1436, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 35, _iter59->first = sent _iter59->second = 22, ] ] ] 2025-02-10 Mon 01:58:42:114.596 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604554069760, Pid 78556]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-02-10 Mon 01:58:42:114.606 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604554069760, Pid 78556]: Processing scm::EvSandeshSend in state Idle 2025-02-10 Mon 01:58:42:114.629 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604554069760, Pid 78556]: SANDESH: Send FAILED: 1739152722114235 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-2705-3 process_status= [ [ [ module_id = contrail-dns instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.17:8086, ] ] status = Down description = ClientInit : EvConnectTimerExpired ], [ type = Database name = Cassandra server_addrs= [ [ (*_iter6) = 10.0.0.17:9041, (*_iter6) = 10.0.0.12:9041, (*_iter6) = 10.0.0.30:9041, ] ] status = Up description = Established Cassandra connection ], [ type = Database name = RabbitMQ server_addrs= [ [ (*_iter6) = 10.0.0.17:5673, (*_iter6) = 10.0.0.12:5673, (*_iter6) = 10.0.0.30:5673, ] ] status = Up description = RabbitMQ connection established ], ] ] flag_infos= [ [ ] ] description = Collector connection down ], ] ] ] 2025-02-10 Mon 01:58:42:114.640 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604554069760, Pid 78556]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-02-10 Mon 01:58:42:114.648 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604554069760, Pid 78556]: Processing scm::EvTcpDeleteSession in state Idle 2025-02-10 Mon 01:58:42:114.685 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604554069760, Pid 78556]: Processing scm::EvSandeshSend in state Idle 2025-02-10 Mon 01:58:42:114.891 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604554069760, Pid 78556]: SANDESH: Send FAILED: 1739152722114359 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-2705-3:Control:contrail-dns:0 client_info= [ status = Idle successful_connections = 4 pid = 78556 http_port = 8092 start_time = 1739152566176088 collector_name = collector_ip = 10.0.0.30:8086 collector_list= [ [ (*_iter40) = 10.0.0.17:8086, (*_iter40) = 10.0.0.30:8086, (*_iter40) = 10.0.0.12:8086, ] ] rx_socket_stats= [ bytes = 2034 calls = 4 average_bytes = 508 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 211827 calls = 18 average_bytes = 11768 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ConfigClientGetRowError [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 6 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigClientMgrDebug [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 647 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigurationMalformedReferenceWarningLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 17 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = DnsOperational [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 14 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapLinkOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 469 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapNodeOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 271 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 6 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 17 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlClientToServer [ messages_sent = 4 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlServerToClient [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 10 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 18 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 10 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveDnsInfo [ messages_sent = 2 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 1436, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 35, _iter59->first = sent _iter59->second = 22, ] ] ] 2025-02-10 Mon 01:58:42:114.908 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604554069760, Pid 78556]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-02-10 Mon 01:58:47:114.425 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604558268160, Pid 78556]: Processing scm::EvIdleHoldTimerExpired in state Idle 2025-02-10 Mon 01:58:47:114.780 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604558268160, Pid 78556]: Connect : Start Connect timer 10.0.0.30:8086 2025-02-10 Mon 01:58:47:114.970 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604558268160, Pid 78556]: Processing scm::EvSandeshSend in state Connect 2025-02-10 Mon 01:58:47:115.012 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604558268160, Pid 78556]: SANDESH: Send FAILED: 1739152727114524 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-2705-3 process_status= [ [ [ module_id = contrail-dns instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.30:8086, ] ] status = Initializing description = Idle : EvIdleHoldTimerExpired -> Connect ], [ type = Database name = Cassandra server_addrs= [ [ (*_iter6) = 10.0.0.17:9041, (*_iter6) = 10.0.0.12:9041, (*_iter6) = 10.0.0.30:9041, ] ] status = Up description = Established Cassandra connection ], [ type = Database name = RabbitMQ server_addrs= [ [ (*_iter6) = 10.0.0.17:5673, (*_iter6) = 10.0.0.12:5673, (*_iter6) = 10.0.0.30:5673, ] ] status = Up description = RabbitMQ connection established ], ] ] flag_infos= [ [ ] ] description = Collector connection down ], ] ] ] 2025-02-10 Mon 01:58:47:115.025 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604558268160, Pid 78556]: Wrong state: Connect for event: EvSandeshSend message: NodeStatusUVE 2025-02-10 Mon 01:58:47:115.050 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604558268160, Pid 78556]: Processing scm::EvSandeshSend in state Connect 2025-02-10 Mon 01:58:47:115.430 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604558268160, Pid 78556]: SANDESH: Send FAILED: 1739152727114944 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-2705-3:Control:contrail-dns:0 client_info= [ status = Connect successful_connections = 4 pid = 78556 http_port = 8092 start_time = 1739152566176088 collector_name = collector_ip = 10.0.0.30:8086 collector_list= [ [ (*_iter40) = 10.0.0.17:8086, (*_iter40) = 10.0.0.30:8086, (*_iter40) = 10.0.0.12:8086, ] ] rx_socket_stats= [ bytes = 2034 calls = 4 average_bytes = 508 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 211827 calls = 18 average_bytes = 11768 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ConfigClientGetRowError [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 6 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigClientMgrDebug [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 647 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigurationMalformedReferenceWarningLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 17 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = DnsOperational [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 14 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapLinkOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 469 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapNodeOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 271 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 6 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 18 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlClientToServer [ messages_sent = 4 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlServerToClient [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 10 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 20 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 10 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveDnsInfo [ messages_sent = 2 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 1436, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 38, _iter59->first = sent _iter59->second = 22, ] ] ] 2025-02-10 Mon 01:58:47:115.454 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604558268160, Pid 78556]: Wrong state: Connect for event: EvSandeshSend message: SandeshModuleClientTrace 2025-02-10 Mon 01:58:47:116.831 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604735187072, Pid 78556]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.0.0.30:49480::10.0.0.30:8086(-1) < Active session connection complete src/contrail-common/io/tcp_session.cc 280 2025-02-10 Mon 01:58:47:116.865 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604735187072, Pid 78556]: 10.0.0.30:49480::10.0.0.30:8086(-1) OnSessionEvent TCP Connected 2025-02-10 Mon 01:58:47:116.878 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604735187072, Pid 78556]: 10.0.0.30:49480::10.0.0.30:8086(-1):EvTcpConnected 2025-02-10 Mon 01:58:47:116.937 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604331771648, Pid 78556]: Processing scm::EvTcpConnected in state Connect 2025-02-10 Mon 01:58:47:116.966 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604331771648, Pid 78556]: Connect : EvTcpConnected : Cancelling Connect timer 2025-02-10 Mon 01:58:47:117.004 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604331771648, Pid 78556]: ClientInit 2025-02-10 Mon 01:58:47:117.022 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604331771648, Pid 78556]: Sending Ctrl Message for cn-jenkins-deploy-platform-ansible-os-2705-3:contrail-dns:0:Control count 5 2025-02-10 Mon 01:58:47:117.124 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604331771648, Pid 78556]: Processing scm::EvSandeshSend in state ClientInit 2025-02-10 Mon 01:58:47:117.134 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604331771648, Pid 78556]: ClientInit : EvSandeshSend : NodeStatusUVE 2025-02-10 Mon 01:58:47:117.172 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604331771648, Pid 78556]: SANDESH: Send FAILED: 1739152727116996 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-2705-3 process_status= [ [ [ module_id = contrail-dns instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.30:8086, ] ] status = Initializing description = Connect : EvTcpConnected ], [ type = Database name = Cassandra server_addrs= [ [ (*_iter6) = 10.0.0.17:9041, (*_iter6) = 10.0.0.12:9041, (*_iter6) = 10.0.0.30:9041, ] ] status = Up description = Established Cassandra connection ], [ type = Database name = RabbitMQ server_addrs= [ [ (*_iter6) = 10.0.0.17:5673, (*_iter6) = 10.0.0.12:5673, (*_iter6) = 10.0.0.30:5673, ] ] status = Up description = RabbitMQ connection established ], ] ] flag_infos= [ [ ] ] description = Collector connection down ], ] ] ] 2025-02-10 Mon 01:58:47:117.190 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604331771648, Pid 78556]: Received UVE message in wrong state : NodeStatusUVE 2025-02-10 Mon 01:58:47:117.198 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604331771648, Pid 78556]: Processing scm::EvSandeshSend in state ClientInit 2025-02-10 Mon 01:58:47:117.205 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604331771648, Pid 78556]: ClientInit : EvSandeshSend : SandeshCtrlClientToServer 2025-02-10 Mon 01:58:47:117.215 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604331771648, Pid 78556]: Processing scm::EvSandeshSend in state ClientInit 2025-02-10 Mon 01:58:47:117.221 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604331771648, Pid 78556]: ClientInit : EvSandeshSend : SandeshModuleClientTrace 2025-02-10 Mon 01:58:47:117.423 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604331771648, Pid 78556]: SANDESH: Send FAILED: 1739152727117111 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-2705-3:Control:contrail-dns:0 client_info= [ status = ClientInit successful_connections = 5 pid = 78556 http_port = 8092 start_time = 1739152566176088 collector_name = collector_ip = 10.0.0.30:8086 collector_list= [ [ (*_iter40) = 10.0.0.17:8086, (*_iter40) = 10.0.0.30:8086, (*_iter40) = 10.0.0.12:8086, ] ] rx_socket_stats= [ bytes = 2034 calls = 4 average_bytes = 508 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 211827 calls = 18 average_bytes = 11768 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ConfigClientGetRowError [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 6 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigClientMgrDebug [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 647 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigurationMalformedReferenceWarningLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 17 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = DnsOperational [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 14 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapLinkOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 469 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapNodeOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 271 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 6 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 19 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlClientToServer [ messages_sent = 4 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlServerToClient [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 10 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 21 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 11 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveDnsInfo [ messages_sent = 2 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 1437, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 40, _iter59->first = sent _iter59->second = 22, ] ] ] 2025-02-10 Mon 01:58:47:117.448 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604331771648, Pid 78556]: Received UVE message in wrong state : SandeshModuleClientTrace 2025-02-10 Mon 01:58:47:445.265 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604735187072, Pid 78556]: TCP [SYS_ERR]: TcpSessionMessageLog: Session 10.0.0.30:49480::10.0.0.30:8086(-1) < Read failed due to error system 104 : Connection reset by peer src/contrail-common/io/tcp_session.cc 507 2025-02-10 Mon 01:58:47:445.309 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604735187072, Pid 78556]: TCP [SYS_ERR]: TcpSessionMessageLog: Session 10.0.0.30:49480::10.0.0.30:8086(-1) > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 312 2025-02-10 Mon 01:58:47:445.330 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604735187072, Pid 78556]: 10.0.0.30:49480::10.0.0.30:8086(-1) OnSessionEvent TCP Connection Closed 2025-02-10 Mon 01:58:47:445.338 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604735187072, Pid 78556]: 10.0.0.30:49480::10.0.0.30:8086(-1):EvTcpClose 2025-02-10 Mon 01:58:47:445.396 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604558268160, Pid 78556]: Processing scm::EvTcpClose in state ClientInit 2025-02-10 Mon 01:58:47:445.407 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604558268160, Pid 78556]: ClientInit : EvTcpClose 2025-02-10 Mon 01:58:47:445.555 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604558268160, Pid 78556]: Return to idle with 5000 2025-02-10 Mon 01:58:47:445.564 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604558268160, Pid 78556]: Idle 2025-02-10 Mon 01:58:47:445.625 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604558268160, Pid 78556]: Processing scm::EvSandeshSend in state Idle 2025-02-10 Mon 01:58:47:445.852 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604558268160, Pid 78556]: SANDESH: Send FAILED: 1739152727445508 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-2705-3:Control:contrail-dns:0 client_info= [ status = ClientInit successful_connections = 5 pid = 78556 http_port = 8092 start_time = 1739152566176088 collector_name = collector_ip = 10.0.0.12:8086 collector_list= [ [ (*_iter40) = 10.0.0.17:8086, (*_iter40) = 10.0.0.30:8086, (*_iter40) = 10.0.0.12:8086, ] ] rx_socket_stats= [ bytes = 2034 calls = 4 average_bytes = 508 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 213437 calls = 19 average_bytes = 11233 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ConfigClientGetRowError [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 6 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigClientMgrDebug [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 647 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigurationMalformedReferenceWarningLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 17 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = DnsOperational [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 14 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapLinkOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 469 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapNodeOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 271 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 6 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 20 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlClientToServer [ messages_sent = 5 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlServerToClient [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 10 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 22 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 13 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveDnsInfo [ messages_sent = 2 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 1439, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 42, _iter59->first = sent _iter59->second = 23, ] ] ] 2025-02-10 Mon 01:58:47:445.872 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604558268160, Pid 78556]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-02-10 Mon 01:58:47:445.882 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604558268160, Pid 78556]: Processing scm::EvSandeshSend in state Idle 2025-02-10 Mon 01:58:47:445.910 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604558268160, Pid 78556]: SANDESH: Send FAILED: 1739152727445542 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-2705-3 process_status= [ [ [ module_id = contrail-dns instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.30:8086, ] ] status = Down description = ClientInit : EvTcpClose ], [ type = Database name = Cassandra server_addrs= [ [ (*_iter6) = 10.0.0.17:9041, (*_iter6) = 10.0.0.12:9041, (*_iter6) = 10.0.0.30:9041, ] ] status = Up description = Established Cassandra connection ], [ type = Database name = RabbitMQ server_addrs= [ [ (*_iter6) = 10.0.0.17:5673, (*_iter6) = 10.0.0.12:5673, (*_iter6) = 10.0.0.30:5673, ] ] status = Up description = RabbitMQ connection established ], ] ] flag_infos= [ [ ] ] description = Collector connection down ], ] ] ] 2025-02-10 Mon 01:58:47:445.919 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604558268160, Pid 78556]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-02-10 Mon 01:58:47:445.928 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604558268160, Pid 78556]: Processing scm::EvTcpDeleteSession in state Idle 2025-02-10 Mon 01:58:47:445.954 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604558268160, Pid 78556]: Processing scm::EvSandeshSend in state Idle 2025-02-10 Mon 01:58:47:446.149 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604558268160, Pid 78556]: SANDESH: Send FAILED: 1739152727445614 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-2705-3:Control:contrail-dns:0 client_info= [ status = Idle successful_connections = 5 pid = 78556 http_port = 8092 start_time = 1739152566176088 collector_name = collector_ip = 10.0.0.12:8086 collector_list= [ [ (*_iter40) = 10.0.0.17:8086, (*_iter40) = 10.0.0.30:8086, (*_iter40) = 10.0.0.12:8086, ] ] rx_socket_stats= [ bytes = 2034 calls = 4 average_bytes = 508 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 213437 calls = 19 average_bytes = 11233 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ConfigClientGetRowError [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 6 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigClientMgrDebug [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 647 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigurationMalformedReferenceWarningLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 17 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = DnsOperational [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 14 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapLinkOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 469 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapNodeOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 271 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 6 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 20 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlClientToServer [ messages_sent = 5 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlServerToClient [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 10 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 22 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 13 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveDnsInfo [ messages_sent = 2 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 1439, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 42, _iter59->first = sent _iter59->second = 23, ] ] ] 2025-02-10 Mon 01:58:47:446.170 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604558268160, Pid 78556]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-02-10 Mon 01:58:52:445.692 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604344366848, Pid 78556]: Processing scm::EvIdleHoldTimerExpired in state Idle 2025-02-10 Mon 01:58:52:445.948 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604344366848, Pid 78556]: Connect : Start Connect timer 10.0.0.12:8086 2025-02-10 Mon 01:58:52:446.092 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604344366848, Pid 78556]: Processing scm::EvSandeshSend in state Connect 2025-02-10 Mon 01:58:52:446.167 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604344366848, Pid 78556]: SANDESH: Send FAILED: 1739152732445779 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-2705-3 process_status= [ [ [ module_id = contrail-dns instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.12:8086, ] ] status = Initializing description = Idle : EvIdleHoldTimerExpired -> Connect ], [ type = Database name = Cassandra server_addrs= [ [ (*_iter6) = 10.0.0.17:9041, (*_iter6) = 10.0.0.12:9041, (*_iter6) = 10.0.0.30:9041, ] ] status = Up description = Established Cassandra connection ], [ type = Database name = RabbitMQ server_addrs= [ [ (*_iter6) = 10.0.0.17:5673, (*_iter6) = 10.0.0.12:5673, (*_iter6) = 10.0.0.30:5673, ] ] status = Up description = RabbitMQ connection established ], ] ] flag_infos= [ [ ] ] description = Collector connection down ], ] ] ] 2025-02-10 Mon 01:58:52:446.180 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604344366848, Pid 78556]: Wrong state: Connect for event: EvSandeshSend message: NodeStatusUVE 2025-02-10 Mon 01:58:52:446.185 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604735187072, Pid 78556]: -(-1) OnSessionEvent TCP Connect Failed 2025-02-10 Mon 01:58:52:446.227 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604735187072, Pid 78556]: -(-1):EvTcpConnectFail 2025-02-10 Mon 01:58:52:446.190 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604344366848, Pid 78556]: Processing scm::EvSandeshSend in state Connect 2025-02-10 Mon 01:58:52:446.453 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604344366848, Pid 78556]: SANDESH: Send FAILED: 1739152732446069 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-2705-3:Control:contrail-dns:0 client_info= [ status = Connect successful_connections = 5 pid = 78556 http_port = 8092 start_time = 1739152566176088 collector_name = collector_ip = 10.0.0.12:8086 collector_list= [ [ (*_iter40) = 10.0.0.17:8086, (*_iter40) = 10.0.0.30:8086, (*_iter40) = 10.0.0.12:8086, ] ] rx_socket_stats= [ bytes = 2034 calls = 4 average_bytes = 508 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 213437 calls = 19 average_bytes = 11233 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ConfigClientGetRowError [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 6 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigClientMgrDebug [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 647 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigurationMalformedReferenceWarningLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 17 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = DnsOperational [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 14 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapLinkOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 469 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapNodeOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 271 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 6 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 21 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlClientToServer [ messages_sent = 5 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlServerToClient [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 10 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 24 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 13 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveDnsInfo [ messages_sent = 2 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 1439, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 45, _iter59->first = sent _iter59->second = 23, ] ] ] 2025-02-10 Mon 01:58:52:446.478 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604344366848, Pid 78556]: Wrong state: Connect for event: EvSandeshSend message: SandeshModuleClientTrace 2025-02-10 Mon 01:58:52:446.488 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604344366848, Pid 78556]: Processing scm::EvTcpConnectFail in state Connect 2025-02-10 Mon 01:58:52:446.496 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604344366848, Pid 78556]: Connect : EvTcpConnectFail 2025-02-10 Mon 01:58:52:446.598 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604344366848, Pid 78556]: TCP [SYS_ERR]: TcpSessionMessageLog: Session -(-1) > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 312 2025-02-10 Mon 01:58:52:446.627 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604344366848, Pid 78556]: Idle 2025-02-10 Mon 01:58:52:446.688 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604344366848, Pid 78556]: Processing scm::EvSandeshSend in state Idle 2025-02-10 Mon 01:58:52:446.910 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604344366848, Pid 78556]: SANDESH: Send FAILED: 1739152732446556 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-2705-3:Control:contrail-dns:0 client_info= [ status = Connect successful_connections = 5 pid = 78556 http_port = 8092 start_time = 1739152566176088 collector_name = collector_ip = 10.0.0.17:8086 collector_list= [ [ (*_iter40) = 10.0.0.17:8086, (*_iter40) = 10.0.0.30:8086, (*_iter40) = 10.0.0.12:8086, ] ] rx_socket_stats= [ bytes = 2034 calls = 4 average_bytes = 508 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 213437 calls = 19 average_bytes = 11233 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ConfigClientGetRowError [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 6 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigClientMgrDebug [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 647 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigurationMalformedReferenceWarningLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 17 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = DnsOperational [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 14 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapLinkOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 469 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapNodeOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 271 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 6 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 22 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlClientToServer [ messages_sent = 5 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlServerToClient [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 10 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 25 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 13 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveDnsInfo [ messages_sent = 2 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 1439, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 47, _iter59->first = sent _iter59->second = 23, ] ] ] 2025-02-10 Mon 01:58:52:446.933 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604344366848, Pid 78556]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-02-10 Mon 01:58:52:446.945 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604344366848, Pid 78556]: Processing scm::EvSandeshSend in state Idle 2025-02-10 Mon 01:58:52:446.965 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604344366848, Pid 78556]: SANDESH: Send FAILED: 1739152732446578 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-2705-3 process_status= [ [ [ module_id = contrail-dns instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], [ type = Database name = Cassandra server_addrs= [ [ (*_iter6) = 10.0.0.17:9041, (*_iter6) = 10.0.0.12:9041, (*_iter6) = 10.0.0.30:9041, ] ] status = Up description = Established Cassandra connection ], [ type = Database name = RabbitMQ server_addrs= [ [ (*_iter6) = 10.0.0.17:5673, (*_iter6) = 10.0.0.12:5673, (*_iter6) = 10.0.0.30:5673, ] ] status = Up description = RabbitMQ connection established ], ] ] flag_infos= [ [ ] ] description = Collector connection down ], ] ] ] 2025-02-10 Mon 01:58:52:446.975 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604344366848, Pid 78556]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-02-10 Mon 01:58:52:446.984 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604344366848, Pid 78556]: Processing scm::EvTcpDeleteSession in state Idle 2025-02-10 Mon 01:58:52:447.005 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604344366848, Pid 78556]: Processing scm::EvSandeshSend in state Idle 2025-02-10 Mon 01:58:52:447.240 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604344366848, Pid 78556]: SANDESH: Send FAILED: 1739152732446676 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-2705-3:Control:contrail-dns:0 client_info= [ status = Idle successful_connections = 5 pid = 78556 http_port = 8092 start_time = 1739152566176088 collector_name = collector_ip = 10.0.0.17:8086 collector_list= [ [ (*_iter40) = 10.0.0.17:8086, (*_iter40) = 10.0.0.30:8086, (*_iter40) = 10.0.0.12:8086, ] ] rx_socket_stats= [ bytes = 2034 calls = 4 average_bytes = 508 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 213437 calls = 19 average_bytes = 11233 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ConfigClientGetRowError [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 6 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigClientMgrDebug [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 647 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigurationMalformedReferenceWarningLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 17 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = DnsOperational [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 14 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapLinkOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 469 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapNodeOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 271 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 6 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 22 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlClientToServer [ messages_sent = 5 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlServerToClient [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 10 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 25 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 14 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveDnsInfo [ messages_sent = 2 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 1440, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 47, _iter59->first = sent _iter59->second = 23, ] ] ] 2025-02-10 Mon 01:58:52:448.103 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604344366848, Pid 78556]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-02-10 Mon 01:59:00:446.769 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604579260160, Pid 78556]: Processing scm::EvIdleHoldTimerExpired in state Idle 2025-02-10 Mon 01:59:00:447.075 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604579260160, Pid 78556]: Connect : Start Connect timer 10.0.0.17:8086 2025-02-10 Mon 01:59:00:447.245 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604579260160, Pid 78556]: Processing scm::EvSandeshSend in state Connect 2025-02-10 Mon 01:59:00:447.291 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604579260160, Pid 78556]: SANDESH: Send FAILED: 1739152740446866 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-2705-3 process_status= [ [ [ module_id = contrail-dns instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.17:8086, ] ] status = Initializing description = Idle : EvIdleHoldTimerExpired -> Connect ], [ type = Database name = Cassandra server_addrs= [ [ (*_iter6) = 10.0.0.17:9041, (*_iter6) = 10.0.0.12:9041, (*_iter6) = 10.0.0.30:9041, ] ] status = Up description = Established Cassandra connection ], [ type = Database name = RabbitMQ server_addrs= [ [ (*_iter6) = 10.0.0.17:5673, (*_iter6) = 10.0.0.12:5673, (*_iter6) = 10.0.0.30:5673, ] ] status = Up description = RabbitMQ connection established ], ] ] flag_infos= [ [ ] ] description = Collector connection down ], ] ] ] 2025-02-10 Mon 01:59:00:447.313 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604579260160, Pid 78556]: Wrong state: Connect for event: EvSandeshSend message: NodeStatusUVE 2025-02-10 Mon 01:59:00:447.327 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604579260160, Pid 78556]: Processing scm::EvSandeshSend in state Connect 2025-02-10 Mon 01:59:00:447.679 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604579260160, Pid 78556]: SANDESH: Send FAILED: 1739152740447224 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-2705-3:Control:contrail-dns:0 client_info= [ status = Connect successful_connections = 5 pid = 78556 http_port = 8092 start_time = 1739152566176088 collector_name = collector_ip = 10.0.0.17:8086 collector_list= [ [ (*_iter40) = 10.0.0.17:8086, (*_iter40) = 10.0.0.30:8086, (*_iter40) = 10.0.0.12:8086, ] ] rx_socket_stats= [ bytes = 2034 calls = 4 average_bytes = 508 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 213437 calls = 19 average_bytes = 11233 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ConfigClientGetRowError [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 6 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigClientMgrDebug [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 647 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigurationMalformedReferenceWarningLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 17 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = DnsOperational [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 14 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapLinkOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 469 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapNodeOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 271 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 6 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 23 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlClientToServer [ messages_sent = 5 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlServerToClient [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 10 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 27 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 14 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveDnsInfo [ messages_sent = 2 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 1440, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 50, _iter59->first = sent _iter59->second = 23, ] ] ] 2025-02-10 Mon 01:59:00:447.709 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604579260160, Pid 78556]: Wrong state: Connect for event: EvSandeshSend message: SandeshModuleClientTrace 2025-02-10 Mon 01:59:00:447.791 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604735187072, Pid 78556]: -(-1) OnSessionEvent TCP Connect Failed 2025-02-10 Mon 01:59:00:447.808 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604735187072, Pid 78556]: -(-1):EvTcpConnectFail 2025-02-10 Mon 01:59:00:447.853 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604554069760, Pid 78556]: Processing scm::EvTcpConnectFail in state Connect 2025-02-10 Mon 01:59:00:447.890 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604554069760, Pid 78556]: Connect : EvTcpConnectFail 2025-02-10 Mon 01:59:00:448.025 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604554069760, Pid 78556]: TCP [SYS_ERR]: TcpSessionMessageLog: Session -(-1) > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 312 2025-02-10 Mon 01:59:00:448.068 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604554069760, Pid 78556]: Idle 2025-02-10 Mon 01:59:00:448.135 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604554069760, Pid 78556]: Processing scm::EvSandeshSend in state Idle 2025-02-10 Mon 01:59:00:448.344 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604554069760, Pid 78556]: SANDESH: Send FAILED: 1739152740447971 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-2705-3:Control:contrail-dns:0 client_info= [ status = Connect successful_connections = 5 pid = 78556 http_port = 8092 start_time = 1739152566176088 collector_name = collector_ip = 10.0.0.30:8086 collector_list= [ [ (*_iter40) = 10.0.0.17:8086, (*_iter40) = 10.0.0.30:8086, (*_iter40) = 10.0.0.12:8086, ] ] rx_socket_stats= [ bytes = 2034 calls = 4 average_bytes = 508 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 213437 calls = 19 average_bytes = 11233 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ConfigClientGetRowError [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 6 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigClientMgrDebug [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 647 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigurationMalformedReferenceWarningLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 17 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = DnsOperational [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 14 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapLinkOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 469 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapNodeOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 271 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 6 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 24 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlClientToServer [ messages_sent = 5 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlServerToClient [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 10 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 28 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 14 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveDnsInfo [ messages_sent = 2 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 1440, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 52, _iter59->first = sent _iter59->second = 23, ] ] ] 2025-02-10 Mon 01:59:00:448.376 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604554069760, Pid 78556]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-02-10 Mon 01:59:00:448.390 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604554069760, Pid 78556]: Processing scm::EvSandeshSend in state Idle 2025-02-10 Mon 01:59:00:448.417 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604554069760, Pid 78556]: SANDESH: Send FAILED: 1739152740447999 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-2705-3 process_status= [ [ [ module_id = contrail-dns instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], [ type = Database name = Cassandra server_addrs= [ [ (*_iter6) = 10.0.0.17:9041, (*_iter6) = 10.0.0.12:9041, (*_iter6) = 10.0.0.30:9041, ] ] status = Up description = Established Cassandra connection ], [ type = Database name = RabbitMQ server_addrs= [ [ (*_iter6) = 10.0.0.17:5673, (*_iter6) = 10.0.0.12:5673, (*_iter6) = 10.0.0.30:5673, ] ] status = Up description = RabbitMQ connection established ], ] ] flag_infos= [ [ ] ] description = Collector connection down ], ] ] ] 2025-02-10 Mon 01:59:00:448.424 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604554069760, Pid 78556]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-02-10 Mon 01:59:00:448.433 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604554069760, Pid 78556]: Processing scm::EvTcpDeleteSession in state Idle 2025-02-10 Mon 01:59:00:448.459 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604554069760, Pid 78556]: Processing scm::EvSandeshSend in state Idle 2025-02-10 Mon 01:59:00:448.649 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604554069760, Pid 78556]: SANDESH: Send FAILED: 1739152740448124 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-2705-3:Control:contrail-dns:0 client_info= [ status = Idle successful_connections = 5 pid = 78556 http_port = 8092 start_time = 1739152566176088 collector_name = collector_ip = 10.0.0.30:8086 collector_list= [ [ (*_iter40) = 10.0.0.17:8086, (*_iter40) = 10.0.0.30:8086, (*_iter40) = 10.0.0.12:8086, ] ] rx_socket_stats= [ bytes = 2034 calls = 4 average_bytes = 508 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 213437 calls = 19 average_bytes = 11233 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ConfigClientGetRowError [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 6 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigClientMgrDebug [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 647 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigurationMalformedReferenceWarningLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 17 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = DnsOperational [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 14 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapLinkOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 469 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapNodeOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 271 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 6 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 24 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlClientToServer [ messages_sent = 5 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlServerToClient [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 10 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 28 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 15 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveDnsInfo [ messages_sent = 2 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 1441, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 52, _iter59->first = sent _iter59->second = 23, ] ] ] 2025-02-10 Mon 01:59:00:448.671 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604554069760, Pid 78556]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-02-10 Mon 01:59:10:287.090 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604335970048, Pid 78556]: Processing scm::EvSandeshSend in state Idle 2025-02-10 Mon 01:59:10:287.199 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604335970048, Pid 78556]: SANDESH: Send FAILED: 1739152750286922 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-2705-3:Control:contrail-dns:0 sm_queue_count = 0 sm_stats= [ ev_stats= [ [ [ event = scm::EvCollectorUpdate enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvConnectTimerExpired enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvIdleHoldTimerExpired enqueues = 4 dequeues = 4 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvSandeshMessageRecv enqueues = 2 dequeues = 2 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvSandeshSend enqueues = 78 dequeues = 78 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvStart enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpClose enqueues = 4 dequeues = 4 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpConnectFail enqueues = 2 dequeues = 2 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpConnected enqueues = 5 dequeues = 5 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpDeleteSession enqueues = 7 dequeues = 7 enqueue_fails = 0 dequeue_fails = 0 ], [ event = enqueues = 105 dequeues = 105 enqueue_fails = 0 dequeue_fails = 0 ], ] ] state = Idle state_since = 1739152740448067 last_state = Idle last_event = scm::EvSandeshSend last_event_at = 1739152740448457 ] max_sm_queue_count = 6 ] 2025-02-10 Mon 01:59:10:287.215 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604335970048, Pid 78556]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-02-10 Mon 01:59:10:287.225 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604335970048, Pid 78556]: Processing scm::EvSandeshSend in state Idle 2025-02-10 Mon 01:59:10:287.436 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604335970048, Pid 78556]: SANDESH: Send FAILED: 1739152750287049 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-2705-3:Control:contrail-dns:0 client_info= [ status = Idle successful_connections = 5 pid = 78556 http_port = 8092 start_time = 1739152566176088 collector_name = collector_ip = 10.0.0.30:8086 collector_list= [ [ (*_iter40) = 10.0.0.17:8086, (*_iter40) = 10.0.0.30:8086, (*_iter40) = 10.0.0.12:8086, ] ] rx_socket_stats= [ bytes = 2034 calls = 4 average_bytes = 508 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 213437 calls = 19 average_bytes = 11233 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ConfigClientGetRowError [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 6 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigClientMgrDebug [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 647 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigurationMalformedReferenceWarningLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 17 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = DnsOperational [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 14 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapLinkOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 469 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapNodeOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 271 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 6 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 25 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlClientToServer [ messages_sent = 5 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlServerToClient [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 10 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 30 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 15 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveDnsInfo [ messages_sent = 2 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 1441, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 55, _iter59->first = sent _iter59->second = 23, ] ] ] 2025-02-10 Mon 01:59:10:287.457 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604335970048, Pid 78556]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-02-10 Mon 01:59:16:448.190 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604575061760, Pid 78556]: Processing scm::EvIdleHoldTimerExpired in state Idle 2025-02-10 Mon 01:59:16:448.486 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604575061760, Pid 78556]: Connect : Start Connect timer 10.0.0.30:8086 2025-02-10 Mon 01:59:16:448.514 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604735187072, Pid 78556]: -(-1) OnSessionEvent TCP Connect Failed 2025-02-10 Mon 01:59:16:448.546 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604735187072, Pid 78556]: -(-1):EvTcpConnectFail 2025-02-10 Mon 01:59:16:448.608 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604575061760, Pid 78556]: Processing scm::EvSandeshSend in state Connect 2025-02-10 Mon 01:59:16:448.639 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604575061760, Pid 78556]: SANDESH: Send FAILED: 1739152756448284 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-2705-3 process_status= [ [ [ module_id = contrail-dns instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.30:8086, ] ] status = Initializing description = Idle : EvIdleHoldTimerExpired -> Connect ], [ type = Database name = Cassandra server_addrs= [ [ (*_iter6) = 10.0.0.17:9041, (*_iter6) = 10.0.0.12:9041, (*_iter6) = 10.0.0.30:9041, ] ] status = Up description = Established Cassandra connection ], [ type = Database name = RabbitMQ server_addrs= [ [ (*_iter6) = 10.0.0.17:5673, (*_iter6) = 10.0.0.12:5673, (*_iter6) = 10.0.0.30:5673, ] ] status = Up description = RabbitMQ connection established ], ] ] flag_infos= [ [ ] ] description = Collector connection down ], ] ] ] 2025-02-10 Mon 01:59:16:448.648 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604575061760, Pid 78556]: Wrong state: Connect for event: EvSandeshSend message: NodeStatusUVE 2025-02-10 Mon 01:59:16:448.657 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604575061760, Pid 78556]: Processing scm::EvTcpConnectFail in state Connect 2025-02-10 Mon 01:59:16:448.664 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604575061760, Pid 78556]: Connect : EvTcpConnectFail 2025-02-10 Mon 01:59:16:448.770 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604575061760, Pid 78556]: TCP [SYS_ERR]: TcpSessionMessageLog: Session -(-1) > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 312 2025-02-10 Mon 01:59:16:448.801 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604575061760, Pid 78556]: Idle 2025-02-10 Mon 01:59:16:448.867 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604575061760, Pid 78556]: Processing scm::EvSandeshSend in state Idle 2025-02-10 Mon 01:59:16:449.087 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604575061760, Pid 78556]: SANDESH: Send FAILED: 1739152756448589 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-2705-3:Control:contrail-dns:0 client_info= [ status = Connect successful_connections = 5 pid = 78556 http_port = 8092 start_time = 1739152566176088 collector_name = collector_ip = 10.0.0.30:8086 collector_list= [ [ (*_iter40) = 10.0.0.17:8086, (*_iter40) = 10.0.0.30:8086, (*_iter40) = 10.0.0.12:8086, ] ] rx_socket_stats= [ bytes = 2034 calls = 4 average_bytes = 508 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 213437 calls = 19 average_bytes = 11233 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ConfigClientGetRowError [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 6 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigClientMgrDebug [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 647 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigurationMalformedReferenceWarningLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 17 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = DnsOperational [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 14 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapLinkOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 469 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapNodeOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 271 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 6 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 25 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlClientToServer [ messages_sent = 5 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlServerToClient [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 10 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 32 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 15 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveDnsInfo [ messages_sent = 2 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 1441, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 57, _iter59->first = sent _iter59->second = 23, ] ] ] 2025-02-10 Mon 01:59:16:449.115 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604575061760, Pid 78556]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-02-10 Mon 01:59:16:449.125 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604575061760, Pid 78556]: Processing scm::EvSandeshSend in state Idle 2025-02-10 Mon 01:59:16:449.337 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604575061760, Pid 78556]: SANDESH: Send FAILED: 1739152756448714 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-2705-3:Control:contrail-dns:0 client_info= [ status = Connect successful_connections = 5 pid = 78556 http_port = 8092 start_time = 1739152566176088 collector_name = collector_ip = 10.0.0.12:8086 collector_list= [ [ (*_iter40) = 10.0.0.17:8086, (*_iter40) = 10.0.0.30:8086, (*_iter40) = 10.0.0.12:8086, ] ] rx_socket_stats= [ bytes = 2034 calls = 4 average_bytes = 508 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 213437 calls = 19 average_bytes = 11233 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ConfigClientGetRowError [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 6 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigClientMgrDebug [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 647 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigurationMalformedReferenceWarningLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 17 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = DnsOperational [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 14 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapLinkOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 469 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapNodeOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 271 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 6 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 26 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlClientToServer [ messages_sent = 5 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlServerToClient [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 10 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 32 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 15 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveDnsInfo [ messages_sent = 2 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 1441, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 58, _iter59->first = sent _iter59->second = 23, ] ] ] 2025-02-10 Mon 01:59:16:449.354 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604575061760, Pid 78556]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-02-10 Mon 01:59:16:449.364 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604575061760, Pid 78556]: Processing scm::EvSandeshSend in state Idle 2025-02-10 Mon 01:59:16:449.386 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604575061760, Pid 78556]: SANDESH: Send FAILED: 1739152756448750 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-2705-3 process_status= [ [ [ module_id = contrail-dns instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], [ type = Database name = Cassandra server_addrs= [ [ (*_iter6) = 10.0.0.17:9041, (*_iter6) = 10.0.0.12:9041, (*_iter6) = 10.0.0.30:9041, ] ] status = Up description = Established Cassandra connection ], [ type = Database name = RabbitMQ server_addrs= [ [ (*_iter6) = 10.0.0.17:5673, (*_iter6) = 10.0.0.12:5673, (*_iter6) = 10.0.0.30:5673, ] ] status = Up description = RabbitMQ connection established ], ] ] flag_infos= [ [ ] ] description = Collector connection down ], ] ] ] 2025-02-10 Mon 01:59:16:449.404 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604575061760, Pid 78556]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-02-10 Mon 01:59:16:449.412 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604575061760, Pid 78556]: Processing scm::EvTcpDeleteSession in state Idle 2025-02-10 Mon 01:59:16:449.430 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604575061760, Pid 78556]: Processing scm::EvSandeshSend in state Idle 2025-02-10 Mon 01:59:16:449.609 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604575061760, Pid 78556]: SANDESH: Send FAILED: 1739152756448855 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-2705-3:Control:contrail-dns:0 client_info= [ status = Idle successful_connections = 5 pid = 78556 http_port = 8092 start_time = 1739152566176088 collector_name = collector_ip = 10.0.0.12:8086 collector_list= [ [ (*_iter40) = 10.0.0.17:8086, (*_iter40) = 10.0.0.30:8086, (*_iter40) = 10.0.0.12:8086, ] ] rx_socket_stats= [ bytes = 2034 calls = 4 average_bytes = 508 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 213437 calls = 19 average_bytes = 11233 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ConfigClientGetRowError [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 6 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigClientMgrDebug [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 647 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigurationMalformedReferenceWarningLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 17 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = DnsOperational [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 14 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapLinkOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 469 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapNodeOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 271 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 6 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 26 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlClientToServer [ messages_sent = 5 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlServerToClient [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 10 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 32 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 16 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveDnsInfo [ messages_sent = 2 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 1442, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 58, _iter59->first = sent _iter59->second = 23, ] ] ] 2025-02-10 Mon 01:59:16:449.629 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604575061760, Pid 78556]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-02-10 Mon 01:59:40:287.338 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604323374848, Pid 78556]: Processing scm::EvSandeshSend in state Idle 2025-02-10 Mon 01:59:40:287.448 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604323374848, Pid 78556]: SANDESH: Send FAILED: 1739152780287161 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-2705-3:Control:contrail-dns:0 sm_queue_count = 0 sm_stats= [ ev_stats= [ [ [ event = scm::EvCollectorUpdate enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvConnectTimerExpired enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvIdleHoldTimerExpired enqueues = 5 dequeues = 5 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvSandeshMessageRecv enqueues = 2 dequeues = 2 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvSandeshSend enqueues = 85 dequeues = 85 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvStart enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpClose enqueues = 4 dequeues = 4 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpConnectFail enqueues = 3 dequeues = 3 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpConnected enqueues = 5 dequeues = 5 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpDeleteSession enqueues = 8 dequeues = 8 enqueue_fails = 0 dequeue_fails = 0 ], [ event = enqueues = 115 dequeues = 115 enqueue_fails = 0 dequeue_fails = 0 ], ] ] state = Idle state_since = 1739152756448800 last_state = Idle last_event = scm::EvSandeshSend last_event_at = 1739152756449429 ] max_sm_queue_count = 6 ] 2025-02-10 Mon 01:59:40:287.469 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604323374848, Pid 78556]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-02-10 Mon 01:59:40:287.480 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604323374848, Pid 78556]: Processing scm::EvSandeshSend in state Idle 2025-02-10 Mon 01:59:40:287.692 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604323374848, Pid 78556]: SANDESH: Send FAILED: 1739152780287281 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-2705-3:Control:contrail-dns:0 client_info= [ status = Idle successful_connections = 5 pid = 78556 http_port = 8092 start_time = 1739152566176088 collector_name = collector_ip = 10.0.0.12:8086 collector_list= [ [ (*_iter40) = 10.0.0.17:8086, (*_iter40) = 10.0.0.30:8086, (*_iter40) = 10.0.0.12:8086, ] ] rx_socket_stats= [ bytes = 2034 calls = 4 average_bytes = 508 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 213437 calls = 19 average_bytes = 11233 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ConfigClientGetRowError [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 6 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigClientMgrDebug [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 647 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigurationMalformedReferenceWarningLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 17 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = DnsOperational [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 14 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapLinkOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 469 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapNodeOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 271 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 6 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 27 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlClientToServer [ messages_sent = 5 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlServerToClient [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 10 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 35 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 16 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveDnsInfo [ messages_sent = 2 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 1442, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 62, _iter59->first = sent _iter59->second = 23, ] ] ] 2025-02-10 Mon 01:59:40:287.716 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604323374848, Pid 78556]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-02-10 Mon 01:59:46:448.895 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604558268160, Pid 78556]: Processing scm::EvIdleHoldTimerExpired in state Idle 2025-02-10 Mon 01:59:46:449.113 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604558268160, Pid 78556]: Connect : Start Connect timer 10.0.0.12:8086 2025-02-10 Mon 01:59:46:449.309 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604735187072, Pid 78556]: -(-1) OnSessionEvent TCP Connect Failed 2025-02-10 Mon 01:59:46:449.321 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604735187072, Pid 78556]: -(-1):EvTcpConnectFail 2025-02-10 Mon 01:59:46:451.523 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604558268160, Pid 78556]: Processing scm::EvSandeshSend in state Connect 2025-02-10 Mon 01:59:46:451.582 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604558268160, Pid 78556]: SANDESH: Send FAILED: 1739152786448982 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-2705-3 process_status= [ [ [ module_id = contrail-dns instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.12:8086, ] ] status = Initializing description = Idle : EvIdleHoldTimerExpired -> Connect ], [ type = Database name = Cassandra server_addrs= [ [ (*_iter6) = 10.0.0.17:9041, (*_iter6) = 10.0.0.12:9041, (*_iter6) = 10.0.0.30:9041, ] ] status = Up description = Established Cassandra connection ], [ type = Database name = RabbitMQ server_addrs= [ [ (*_iter6) = 10.0.0.17:5673, (*_iter6) = 10.0.0.12:5673, (*_iter6) = 10.0.0.30:5673, ] ] status = Up description = RabbitMQ connection established ], ] ] flag_infos= [ [ ] ] description = Collector connection down ], ] ] ] 2025-02-10 Mon 01:59:46:451.596 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604558268160, Pid 78556]: Wrong state: Connect for event: EvSandeshSend message: NodeStatusUVE 2025-02-10 Mon 01:59:46:451.610 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604558268160, Pid 78556]: Processing scm::EvTcpConnectFail in state Connect 2025-02-10 Mon 01:59:46:451.620 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604558268160, Pid 78556]: Connect : EvTcpConnectFail 2025-02-10 Mon 01:59:46:451.754 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604558268160, Pid 78556]: TCP [SYS_ERR]: TcpSessionMessageLog: Session -(-1) > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 312 2025-02-10 Mon 01:59:46:451.805 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604558268160, Pid 78556]: Idle 2025-02-10 Mon 01:59:46:451.905 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604558268160, Pid 78556]: Processing scm::EvSandeshSend in state Idle 2025-02-10 Mon 01:59:46:452.245 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604558268160, Pid 78556]: SANDESH: Send FAILED: 1739152786451497 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-2705-3:Control:contrail-dns:0 client_info= [ status = Connect successful_connections = 5 pid = 78556 http_port = 8092 start_time = 1739152566176088 collector_name = collector_ip = 10.0.0.12:8086 collector_list= [ [ (*_iter40) = 10.0.0.17:8086, (*_iter40) = 10.0.0.30:8086, (*_iter40) = 10.0.0.12:8086, ] ] rx_socket_stats= [ bytes = 2034 calls = 4 average_bytes = 508 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 213437 calls = 19 average_bytes = 11233 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ConfigClientGetRowError [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 6 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigClientMgrDebug [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 647 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigurationMalformedReferenceWarningLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 17 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = DnsOperational [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 14 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapLinkOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 469 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapNodeOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 271 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 6 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 27 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlClientToServer [ messages_sent = 5 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlServerToClient [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 10 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 37 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 16 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveDnsInfo [ messages_sent = 2 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 1442, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 64, _iter59->first = sent _iter59->second = 23, ] ] ] 2025-02-10 Mon 01:59:46:452.276 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604558268160, Pid 78556]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-02-10 Mon 01:59:46:452.292 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604558268160, Pid 78556]: Processing scm::EvSandeshSend in state Idle 2025-02-10 Mon 01:59:46:452.592 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604558268160, Pid 78556]: SANDESH: Send FAILED: 1739152786451696 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-2705-3:Control:contrail-dns:0 client_info= [ status = Connect successful_connections = 5 pid = 78556 http_port = 8092 start_time = 1739152566176088 collector_name = collector_ip = 10.0.0.17:8086 collector_list= [ [ (*_iter40) = 10.0.0.17:8086, (*_iter40) = 10.0.0.30:8086, (*_iter40) = 10.0.0.12:8086, ] ] rx_socket_stats= [ bytes = 2034 calls = 4 average_bytes = 508 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 213437 calls = 19 average_bytes = 11233 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ConfigClientGetRowError [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 6 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigClientMgrDebug [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 647 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigurationMalformedReferenceWarningLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 17 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = DnsOperational [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 14 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapLinkOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 469 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapNodeOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 271 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 6 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 28 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlClientToServer [ messages_sent = 5 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlServerToClient [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 10 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 37 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 16 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveDnsInfo [ messages_sent = 2 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 1442, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 65, _iter59->first = sent _iter59->second = 23, ] ] ] 2025-02-10 Mon 01:59:46:452.614 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604558268160, Pid 78556]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-02-10 Mon 01:59:46:452.629 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604558268160, Pid 78556]: Processing scm::EvSandeshSend in state Idle 2025-02-10 Mon 01:59:46:452.660 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604558268160, Pid 78556]: SANDESH: Send FAILED: 1739152786451729 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-2705-3 process_status= [ [ [ module_id = contrail-dns instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], [ type = Database name = Cassandra server_addrs= [ [ (*_iter6) = 10.0.0.17:9041, (*_iter6) = 10.0.0.12:9041, (*_iter6) = 10.0.0.30:9041, ] ] status = Up description = Established Cassandra connection ], [ type = Database name = RabbitMQ server_addrs= [ [ (*_iter6) = 10.0.0.17:5673, (*_iter6) = 10.0.0.12:5673, (*_iter6) = 10.0.0.30:5673, ] ] status = Up description = RabbitMQ connection established ], ] ] flag_infos= [ [ ] ] description = Collector connection down ], ] ] ] 2025-02-10 Mon 01:59:46:452.674 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604558268160, Pid 78556]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-02-10 Mon 01:59:46:452.688 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604558268160, Pid 78556]: Processing scm::EvTcpDeleteSession in state Idle 2025-02-10 Mon 01:59:46:452.714 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604558268160, Pid 78556]: Processing scm::EvSandeshSend in state Idle 2025-02-10 Mon 01:59:46:453.028 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604558268160, Pid 78556]: SANDESH: Send FAILED: 1739152786451889 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-2705-3:Control:contrail-dns:0 client_info= [ status = Idle successful_connections = 5 pid = 78556 http_port = 8092 start_time = 1739152566176088 collector_name = collector_ip = 10.0.0.17:8086 collector_list= [ [ (*_iter40) = 10.0.0.17:8086, (*_iter40) = 10.0.0.30:8086, (*_iter40) = 10.0.0.12:8086, ] ] rx_socket_stats= [ bytes = 2034 calls = 4 average_bytes = 508 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 213437 calls = 19 average_bytes = 11233 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ConfigClientGetRowError [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 6 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigClientMgrDebug [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 647 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigurationMalformedReferenceWarningLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 17 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = DnsOperational [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 14 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapLinkOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 469 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapNodeOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 271 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 6 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 28 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlClientToServer [ messages_sent = 5 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlServerToClient [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 10 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 37 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 17 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveDnsInfo [ messages_sent = 2 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 1443, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 65, _iter59->first = sent _iter59->second = 23, ] ] ] 2025-02-10 Mon 01:59:46:453.052 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604558268160, Pid 78556]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-02-10 Mon 01:59:50:484.111 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604575061760, Pid 78556]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 571 2025-02-10 Mon 01:59:50:552.953 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604323374848, Pid 78556]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 571 2025-02-10 Mon 01:59:50:725.072 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604579260160, Pid 78556]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 571 2025-02-10 Mon 01:59:50:824.994 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604327573248, Pid 78556]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 571 2025-02-10 Mon 01:59:51:306.690 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604344366848, Pid 78556]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 571 2025-02-10 Mon 01:59:51:329.420 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604327573248, Pid 78556]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 571 2025-02-10 Mon 01:59:56:949.856 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604331771648, Pid 78556]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 571 2025-02-10 Mon 02:00:02:673.951 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604335970048, Pid 78556]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 571 2025-02-10 Mon 02:00:02:681.970 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604554069760, Pid 78556]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 571 2025-02-10 Mon 02:00:02:702.992 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604558268160, Pid 78556]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 571 2025-02-10 Mon 02:00:04:568.535 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604554069760, Pid 78556]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 571 2025-02-10 Mon 02:00:04:599.712 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604579260160, Pid 78556]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 571 2025-02-10 Mon 02:00:04:621.133 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604558268160, Pid 78556]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 571 2025-02-10 Mon 02:00:04:740.593 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604327573248, Pid 78556]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 571 2025-02-10 Mon 02:00:04:769.434 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604570863360, Pid 78556]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 571 2025-02-10 Mon 02:00:04:803.160 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604579260160, Pid 78556]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 571 2025-02-10 Mon 02:00:04:911.008 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604562466560, Pid 78556]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 571 2025-02-10 Mon 02:00:04:924.220 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604323374848, Pid 78556]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 571 2025-02-10 Mon 02:00:04:982.770 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604575061760, Pid 78556]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 571 2025-02-10 Mon 02:00:05:079.239 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604579260160, Pid 78556]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 571 2025-02-10 Mon 02:00:05:108.085 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604327573248, Pid 78556]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 571 2025-02-10 Mon 02:00:05:143.314 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604327573248, Pid 78556]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 571 2025-02-10 Mon 02:00:05:257.283 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604558268160, Pid 78556]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 571 2025-02-10 Mon 02:00:05:280.259 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604344366848, Pid 78556]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 571 2025-02-10 Mon 02:00:05:309.505 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604335970048, Pid 78556]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 571 2025-02-10 Mon 02:00:05:425.392 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604570863360, Pid 78556]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 571 2025-02-10 Mon 02:00:05:445.312 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604570863360, Pid 78556]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 571 2025-02-10 Mon 02:00:05:493.535 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604554069760, Pid 78556]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 571 2025-02-10 Mon 02:00:05:591.631 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604335970048, Pid 78556]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 571 2025-02-10 Mon 02:00:05:627.702 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604579260160, Pid 78556]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 571 2025-02-10 Mon 02:00:05:670.067 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604579260160, Pid 78556]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 571 2025-02-10 Mon 02:00:05:759.725 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604570863360, Pid 78556]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 571 2025-02-10 Mon 02:00:05:813.512 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604575061760, Pid 78556]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 571 2025-02-10 Mon 02:00:05:841.148 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604327573248, Pid 78556]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 571 2025-02-10 Mon 02:00:05:933.142 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604579260160, Pid 78556]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 571 2025-02-10 Mon 02:00:05:968.161 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604575061760, Pid 78556]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 571 2025-02-10 Mon 02:00:05:999.702 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604575061760, Pid 78556]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 571 2025-02-10 Mon 02:00:06:089.541 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604575061760, Pid 78556]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 571 2025-02-10 Mon 02:00:06:135.060 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604554069760, Pid 78556]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 571 2025-02-10 Mon 02:00:06:158.741 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604327573248, Pid 78556]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 571 2025-02-10 Mon 02:00:06:257.986 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604331771648, Pid 78556]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 571 2025-02-10 Mon 02:00:06:303.920 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604562466560, Pid 78556]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 571 2025-02-10 Mon 02:00:06:358.764 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604323374848, Pid 78556]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 571 2025-02-10 Mon 02:00:06:464.342 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604344366848, Pid 78556]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 571 2025-02-10 Mon 02:00:06:473.991 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604575061760, Pid 78556]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 571 2025-02-10 Mon 02:00:06:531.512 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604335970048, Pid 78556]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 571 2025-02-10 Mon 02:00:06:639.117 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604335970048, Pid 78556]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 571 2025-02-10 Mon 02:00:06:652.686 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604335970048, Pid 78556]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 571 2025-02-10 Mon 02:00:06:700.285 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604327573248, Pid 78556]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 571 2025-02-10 Mon 02:00:06:803.179 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604558268160, Pid 78556]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 571 2025-02-10 Mon 02:00:06:812.042 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604331771648, Pid 78556]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 571 2025-02-10 Mon 02:00:06:876.111 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604558268160, Pid 78556]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 571 2025-02-10 Mon 02:00:06:970.684 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604554069760, Pid 78556]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 571 2025-02-10 Mon 02:00:06:979.477 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604340168448, Pid 78556]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 571 2025-02-10 Mon 02:00:07:040.246 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604340168448, Pid 78556]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 571 2025-02-10 Mon 02:00:07:126.244 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604344366848, Pid 78556]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 571 2025-02-10 Mon 02:00:07:161.322 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604344366848, Pid 78556]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 571 2025-02-10 Mon 02:00:07:220.250 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604558268160, Pid 78556]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 571 2025-02-10 Mon 02:00:07:296.001 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604579260160, Pid 78556]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 571 2025-02-10 Mon 02:00:07:331.154 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604554069760, Pid 78556]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 571 2025-02-10 Mon 02:00:07:386.356 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604331771648, Pid 78556]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 571 2025-02-10 Mon 02:00:07:460.871 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604340168448, Pid 78556]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 571 2025-02-10 Mon 02:00:07:504.310 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604331771648, Pid 78556]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 571 2025-02-10 Mon 02:00:07:550.301 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604570863360, Pid 78556]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 571 2025-02-10 Mon 02:00:07:628.656 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604579260160, Pid 78556]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 571 2025-02-10 Mon 02:00:07:649.503 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604344366848, Pid 78556]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 571 2025-02-10 Mon 02:00:07:720.195 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604340168448, Pid 78556]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 571 2025-02-10 Mon 02:00:07:787.418 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604562466560, Pid 78556]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 571 2025-02-10 Mon 02:00:07:816.204 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604344366848, Pid 78556]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 571 2025-02-10 Mon 02:00:07:881.552 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604579260160, Pid 78556]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 571 2025-02-10 Mon 02:00:07:948.809 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604558268160, Pid 78556]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 571 2025-02-10 Mon 02:00:07:974.679 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604562466560, Pid 78556]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 571 2025-02-10 Mon 02:00:08:046.937 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604340168448, Pid 78556]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 571 2025-02-10 Mon 02:00:08:108.609 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604575061760, Pid 78556]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 571 2025-02-10 Mon 02:00:08:144.052 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604554069760, Pid 78556]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 571 2025-02-10 Mon 02:00:08:213.119 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604554069760, Pid 78556]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 571 2025-02-10 Mon 02:00:08:279.169 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604331771648, Pid 78556]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 571 2025-02-10 Mon 02:00:08:299.000 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604570863360, Pid 78556]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 571 2025-02-10 Mon 02:00:08:338.606 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604735187072, Pid 78556]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.20.0.8:8092::10.20.0.6:35558 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 254 2025-02-10 Mon 02:00:08:369.282 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604562466560, Pid 78556]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 571 2025-02-10 Mon 02:00:09:844.982 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604735187072, Pid 78556]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.20.0.8:8092::10.20.0.8:40794 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 254 2025-02-10 Mon 02:00:09:869.661 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604735187072, Pid 78556]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.20.0.8:8092::10.20.0.11:36324 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 254 2025-02-10 Mon 02:00:10:287.552 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604579260160, Pid 78556]: Processing scm::EvSandeshSend in state Idle 2025-02-10 Mon 02:00:10:287.680 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604579260160, Pid 78556]: SANDESH: Send FAILED: 1739152810287395 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-2705-3:Control:contrail-dns:0 sm_queue_count = 0 sm_stats= [ ev_stats= [ [ [ event = scm::EvCollectorUpdate enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvConnectTimerExpired enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvIdleHoldTimerExpired enqueues = 6 dequeues = 6 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvSandeshMessageRecv enqueues = 2 dequeues = 2 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvSandeshSend enqueues = 92 dequeues = 92 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvStart enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpClose enqueues = 4 dequeues = 4 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpConnectFail enqueues = 4 dequeues = 4 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpConnected enqueues = 5 dequeues = 5 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpDeleteSession enqueues = 9 dequeues = 9 enqueue_fails = 0 dequeue_fails = 0 ], [ event = enqueues = 125 dequeues = 125 enqueue_fails = 0 dequeue_fails = 0 ], ] ] state = Idle state_since = 1739152786451803 last_state = Idle last_event = scm::EvSandeshSend last_event_at = 1739152786452712 ] max_sm_queue_count = 6 ] 2025-02-10 Mon 02:00:10:287.696 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604579260160, Pid 78556]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-02-10 Mon 02:00:10:287.707 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604579260160, Pid 78556]: Processing scm::EvSandeshSend in state Idle 2025-02-10 Mon 02:00:10:287.926 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604579260160, Pid 78556]: SANDESH: Send FAILED: 1739152810287511 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-2705-3:Control:contrail-dns:0 client_info= [ status = Idle successful_connections = 5 pid = 78556 http_port = 8092 start_time = 1739152566176088 collector_name = collector_ip = 10.0.0.17:8086 collector_list= [ [ (*_iter40) = 10.0.0.17:8086, (*_iter40) = 10.0.0.30:8086, (*_iter40) = 10.0.0.12:8086, ] ] rx_socket_stats= [ bytes = 2034 calls = 4 average_bytes = 508 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 213437 calls = 19 average_bytes = 11233 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ConfigClientGetRowError [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 6 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigClientMgrDebug [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 726 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigurationMalformedReferenceWarningLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 17 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = DnsOperational [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 14 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapLinkOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 469 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapNodeOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 271 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 6 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 29 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlClientToServer [ messages_sent = 5 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlServerToClient [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 10 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 40 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 20 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveDnsInfo [ messages_sent = 2 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 1525, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 69, _iter59->first = sent _iter59->second = 23, ] ] ] 2025-02-10 Mon 02:00:10:287.952 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604579260160, Pid 78556]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-02-10 Mon 02:00:16:451.943 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604570863360, Pid 78556]: Processing scm::EvIdleHoldTimerExpired in state Idle 2025-02-10 Mon 02:00:16:452.188 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604570863360, Pid 78556]: Connect : Start Connect timer 10.0.0.17:8086 2025-02-10 Mon 02:00:16:452.303 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604735187072, Pid 78556]: -(-1) OnSessionEvent TCP Connect Failed 2025-02-10 Mon 02:00:16:452.339 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604735187072, Pid 78556]: -(-1):EvTcpConnectFail 2025-02-10 Mon 02:00:16:452.307 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604570863360, Pid 78556]: Processing scm::EvSandeshSend in state Connect 2025-02-10 Mon 02:00:16:452.400 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604570863360, Pid 78556]: SANDESH: Send FAILED: 1739152816452026 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-2705-3 process_status= [ [ [ module_id = contrail-dns instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.17:8086, ] ] status = Initializing description = Idle : EvIdleHoldTimerExpired -> Connect ], [ type = Database name = Cassandra server_addrs= [ [ (*_iter6) = 10.0.0.17:9041, (*_iter6) = 10.0.0.12:9041, (*_iter6) = 10.0.0.30:9041, ] ] status = Up description = Established Cassandra connection ], [ type = Database name = RabbitMQ server_addrs= [ [ (*_iter6) = 10.0.0.17:5673, (*_iter6) = 10.0.0.12:5673, (*_iter6) = 10.0.0.30:5673, ] ] status = Up description = RabbitMQ connection established ], ] ] flag_infos= [ [ ] ] description = Collector connection down ], ] ] ] 2025-02-10 Mon 02:00:16:452.410 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604570863360, Pid 78556]: Wrong state: Connect for event: EvSandeshSend message: NodeStatusUVE 2025-02-10 Mon 02:00:16:452.421 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604570863360, Pid 78556]: Processing scm::EvSandeshSend in state Connect 2025-02-10 Mon 02:00:16:452.630 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604570863360, Pid 78556]: SANDESH: Send FAILED: 1739152816452291 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-2705-3:Control:contrail-dns:0 client_info= [ status = Connect successful_connections = 5 pid = 78556 http_port = 8092 start_time = 1739152566176088 collector_name = collector_ip = 10.0.0.17:8086 collector_list= [ [ (*_iter40) = 10.0.0.17:8086, (*_iter40) = 10.0.0.30:8086, (*_iter40) = 10.0.0.12:8086, ] ] rx_socket_stats= [ bytes = 2034 calls = 4 average_bytes = 508 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 213437 calls = 19 average_bytes = 11233 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ConfigClientGetRowError [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 6 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigClientMgrDebug [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 726 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigurationMalformedReferenceWarningLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 17 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = DnsOperational [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 14 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapLinkOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 469 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapNodeOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 271 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 6 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 29 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlClientToServer [ messages_sent = 5 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlServerToClient [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 10 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 42 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 20 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveDnsInfo [ messages_sent = 2 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 1525, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 71, _iter59->first = sent _iter59->second = 23, ] ] ] 2025-02-10 Mon 02:00:16:452.654 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604570863360, Pid 78556]: Wrong state: Connect for event: EvSandeshSend message: SandeshModuleClientTrace 2025-02-10 Mon 02:00:16:452.664 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604570863360, Pid 78556]: Processing scm::EvTcpConnectFail in state Connect 2025-02-10 Mon 02:00:16:452.672 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604570863360, Pid 78556]: Connect : EvTcpConnectFail 2025-02-10 Mon 02:00:16:452.778 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604570863360, Pid 78556]: TCP [SYS_ERR]: TcpSessionMessageLog: Session -(-1) > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 312 2025-02-10 Mon 02:00:16:452.843 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604570863360, Pid 78556]: Idle 2025-02-10 Mon 02:00:16:452.933 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604570863360, Pid 78556]: Processing scm::EvSandeshSend in state Idle 2025-02-10 Mon 02:00:16:453.309 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604570863360, Pid 78556]: SANDESH: Send FAILED: 1739152816452733 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-2705-3:Control:contrail-dns:0 client_info= [ status = Connect successful_connections = 5 pid = 78556 http_port = 8092 start_time = 1739152566176088 collector_name = collector_ip = 10.0.0.30:8086 collector_list= [ [ (*_iter40) = 10.0.0.17:8086, (*_iter40) = 10.0.0.30:8086, (*_iter40) = 10.0.0.12:8086, ] ] rx_socket_stats= [ bytes = 2034 calls = 4 average_bytes = 508 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 213437 calls = 19 average_bytes = 11233 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ConfigClientGetRowError [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 6 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigClientMgrDebug [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 726 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigurationMalformedReferenceWarningLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 17 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = DnsOperational [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 14 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapLinkOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 469 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapNodeOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 271 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 6 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 30 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlClientToServer [ messages_sent = 5 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlServerToClient [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 10 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 43 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 20 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveDnsInfo [ messages_sent = 2 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 1525, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 73, _iter59->first = sent _iter59->second = 23, ] ] ] 2025-02-10 Mon 02:00:16:453.334 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604570863360, Pid 78556]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-02-10 Mon 02:00:16:453.349 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604570863360, Pid 78556]: Processing scm::EvSandeshSend in state Idle 2025-02-10 Mon 02:00:16:453.378 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604570863360, Pid 78556]: SANDESH: Send FAILED: 1739152816452756 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-2705-3 process_status= [ [ [ module_id = contrail-dns instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], [ type = Database name = Cassandra server_addrs= [ [ (*_iter6) = 10.0.0.17:9041, (*_iter6) = 10.0.0.12:9041, (*_iter6) = 10.0.0.30:9041, ] ] status = Up description = Established Cassandra connection ], [ type = Database name = RabbitMQ server_addrs= [ [ (*_iter6) = 10.0.0.17:5673, (*_iter6) = 10.0.0.12:5673, (*_iter6) = 10.0.0.30:5673, ] ] status = Up description = RabbitMQ connection established ], ] ] flag_infos= [ [ ] ] description = Collector connection down ], ] ] ] 2025-02-10 Mon 02:00:16:453.390 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604570863360, Pid 78556]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-02-10 Mon 02:00:16:453.402 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604570863360, Pid 78556]: Processing scm::EvTcpDeleteSession in state Idle 2025-02-10 Mon 02:00:16:453.428 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604570863360, Pid 78556]: Processing scm::EvSandeshSend in state Idle 2025-02-10 Mon 02:00:16:453.730 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604570863360, Pid 78556]: SANDESH: Send FAILED: 1739152816452916 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-2705-3:Control:contrail-dns:0 client_info= [ status = Idle successful_connections = 5 pid = 78556 http_port = 8092 start_time = 1739152566176088 collector_name = collector_ip = 10.0.0.30:8086 collector_list= [ [ (*_iter40) = 10.0.0.17:8086, (*_iter40) = 10.0.0.30:8086, (*_iter40) = 10.0.0.12:8086, ] ] rx_socket_stats= [ bytes = 2034 calls = 4 average_bytes = 508 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 213437 calls = 19 average_bytes = 11233 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ConfigClientGetRowError [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 6 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigClientMgrDebug [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 726 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigurationMalformedReferenceWarningLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 17 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = DnsOperational [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 14 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapLinkOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 469 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapNodeOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 271 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 6 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 30 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlClientToServer [ messages_sent = 5 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlServerToClient [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 10 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 43 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 21 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveDnsInfo [ messages_sent = 2 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 1526, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 73, _iter59->first = sent _iter59->second = 23, ] ] ] 2025-02-10 Mon 02:00:16:453.750 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604570863360, Pid 78556]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-02-10 Mon 02:00:16:925.589 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604562466560, Pid 78556]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 571 2025-02-10 Mon 02:00:16:955.032 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604575061760, Pid 78556]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 571 2025-02-10 Mon 02:00:17:267.725 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604562466560, Pid 78556]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 571 2025-02-10 Mon 02:00:26:504.243 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604331771648, Pid 78556]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 571 2025-02-10 Mon 02:00:26:523.565 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604575061760, Pid 78556]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 571 2025-02-10 Mon 02:00:26:542.618 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604562466560, Pid 78556]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 571 2025-02-10 Mon 02:00:28:692.602 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604575061760, Pid 78556]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 571 2025-02-10 Mon 02:00:28:707.524 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604340168448, Pid 78556]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 571 2025-02-10 Mon 02:00:28:770.589 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604562466560, Pid 78556]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 571 2025-02-10 Mon 02:00:35:055.027 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604735187072, Pid 78556]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.20.0.8:8092::10.20.0.8:57394 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 254 2025-02-10 Mon 02:00:35:616.536 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604735187072, Pid 78556]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.20.0.8:8092::10.20.0.6:45762 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 254 2025-02-10 Mon 02:00:36:487.376 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604735187072, Pid 78556]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.20.0.8:8092::10.20.0.11:48272 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 254 2025-02-10 Mon 02:00:40:288.332 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604323374848, Pid 78556]: Processing scm::EvSandeshSend in state Idle 2025-02-10 Mon 02:00:40:288.443 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604323374848, Pid 78556]: SANDESH: Send FAILED: 1739152840288175 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-2705-3:Control:contrail-dns:0 sm_queue_count = 0 sm_stats= [ ev_stats= [ [ [ event = scm::EvCollectorUpdate enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvConnectTimerExpired enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvIdleHoldTimerExpired enqueues = 7 dequeues = 7 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvSandeshMessageRecv enqueues = 2 dequeues = 2 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvSandeshSend enqueues = 99 dequeues = 99 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvStart enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpClose enqueues = 4 dequeues = 4 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpConnectFail enqueues = 5 dequeues = 5 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpConnected enqueues = 5 dequeues = 5 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpDeleteSession enqueues = 10 dequeues = 10 enqueue_fails = 0 dequeue_fails = 0 ], [ event = enqueues = 135 dequeues = 135 enqueue_fails = 0 dequeue_fails = 0 ], ] ] state = Idle state_since = 1739152816452841 last_state = Idle last_event = scm::EvSandeshSend last_event_at = 1739152816453426 ] max_sm_queue_count = 6 ] 2025-02-10 Mon 02:00:40:288.470 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604323374848, Pid 78556]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-02-10 Mon 02:00:40:288.479 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604323374848, Pid 78556]: Processing scm::EvSandeshSend in state Idle 2025-02-10 Mon 02:00:40:288.676 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604323374848, Pid 78556]: SANDESH: Send FAILED: 1739152840288300 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-2705-3:Control:contrail-dns:0 client_info= [ status = Idle successful_connections = 5 pid = 78556 http_port = 8092 start_time = 1739152566176088 collector_name = collector_ip = 10.0.0.30:8086 collector_list= [ [ (*_iter40) = 10.0.0.17:8086, (*_iter40) = 10.0.0.30:8086, (*_iter40) = 10.0.0.12:8086, ] ] rx_socket_stats= [ bytes = 2034 calls = 4 average_bytes = 508 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 213437 calls = 19 average_bytes = 11233 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ConfigClientGetRowError [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 6 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigClientMgrDebug [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 735 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigurationMalformedReferenceWarningLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 17 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = DnsOperational [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 14 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapLinkOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 469 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapNodeOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 271 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 6 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 31 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlClientToServer [ messages_sent = 5 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlServerToClient [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 10 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 45 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 24 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveDnsInfo [ messages_sent = 2 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 1538, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 76, _iter59->first = sent _iter59->second = 23, ] ] ] 2025-02-10 Mon 02:00:40:288.693 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604323374848, Pid 78556]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-02-10 Mon 02:00:46:453.455 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604570863360, Pid 78556]: Processing scm::EvIdleHoldTimerExpired in state Idle 2025-02-10 Mon 02:00:46:453.659 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604570863360, Pid 78556]: Connect : Start Connect timer 10.0.0.30:8086 2025-02-10 Mon 02:00:46:453.774 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604570863360, Pid 78556]: Processing scm::EvSandeshSend in state Connect 2025-02-10 Mon 02:00:46:453.807 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604570863360, Pid 78556]: SANDESH: Send FAILED: 1739152846453532 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-2705-3 process_status= [ [ [ module_id = contrail-dns instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.30:8086, ] ] status = Initializing description = Idle : EvIdleHoldTimerExpired -> Connect ], [ type = Database name = Cassandra server_addrs= [ [ (*_iter6) = 10.0.0.17:9041, (*_iter6) = 10.0.0.12:9041, (*_iter6) = 10.0.0.30:9041, ] ] status = Up description = Established Cassandra connection ], [ type = Database name = RabbitMQ server_addrs= [ [ (*_iter6) = 10.0.0.17:5673, (*_iter6) = 10.0.0.12:5673, (*_iter6) = 10.0.0.30:5673, ] ] status = Up description = RabbitMQ connection established ], ] ] flag_infos= [ [ ] ] description = Collector connection down ], ] ] ] 2025-02-10 Mon 02:00:46:453.816 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604570863360, Pid 78556]: Wrong state: Connect for event: EvSandeshSend message: NodeStatusUVE 2025-02-10 Mon 02:00:46:453.836 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604570863360, Pid 78556]: Processing scm::EvSandeshSend in state Connect 2025-02-10 Mon 02:00:46:454.052 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604570863360, Pid 78556]: SANDESH: Send FAILED: 1739152846453758 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-2705-3:Control:contrail-dns:0 client_info= [ status = Connect successful_connections = 5 pid = 78556 http_port = 8092 start_time = 1739152566176088 collector_name = collector_ip = 10.0.0.30:8086 collector_list= [ [ (*_iter40) = 10.0.0.17:8086, (*_iter40) = 10.0.0.30:8086, (*_iter40) = 10.0.0.12:8086, ] ] rx_socket_stats= [ bytes = 2034 calls = 4 average_bytes = 508 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 213437 calls = 19 average_bytes = 11233 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ConfigClientGetRowError [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 6 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigClientMgrDebug [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 735 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigurationMalformedReferenceWarningLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 17 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = DnsOperational [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 14 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapLinkOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 469 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapNodeOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 271 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 6 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 31 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlClientToServer [ messages_sent = 5 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlServerToClient [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 10 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 47 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 24 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveDnsInfo [ messages_sent = 2 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 1538, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 78, _iter59->first = sent _iter59->second = 23, ] ] ] 2025-02-10 Mon 02:00:46:454.078 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604570863360, Pid 78556]: Wrong state: Connect for event: EvSandeshSend message: SandeshModuleClientTrace 2025-02-10 Mon 02:00:46:457.204 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604735187072, Pid 78556]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.0.0.30:37998::10.0.0.30:8086(-1) < Active session connection complete src/contrail-common/io/tcp_session.cc 280 2025-02-10 Mon 02:00:46:457.231 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604735187072, Pid 78556]: 10.0.0.30:37998::10.0.0.30:8086(-1) OnSessionEvent TCP Connected 2025-02-10 Mon 02:00:46:457.238 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604735187072, Pid 78556]: 10.0.0.30:37998::10.0.0.30:8086(-1):EvTcpConnected 2025-02-10 Mon 02:00:46:458.554 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604327573248, Pid 78556]: Processing scm::EvTcpConnected in state Connect 2025-02-10 Mon 02:00:46:458.571 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604327573248, Pid 78556]: Connect : EvTcpConnected : Cancelling Connect timer 2025-02-10 Mon 02:00:46:458.607 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604327573248, Pid 78556]: ClientInit 2025-02-10 Mon 02:00:46:458.620 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604327573248, Pid 78556]: Sending Ctrl Message for cn-jenkins-deploy-platform-ansible-os-2705-3:contrail-dns:0:Control count 6 2025-02-10 Mon 02:00:46:458.713 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604327573248, Pid 78556]: Processing scm::EvSandeshSend in state ClientInit 2025-02-10 Mon 02:00:46:458.722 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604327573248, Pid 78556]: ClientInit : EvSandeshSend : NodeStatusUVE 2025-02-10 Mon 02:00:46:458.745 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604327573248, Pid 78556]: SANDESH: Send FAILED: 1739152846458599 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-2705-3 process_status= [ [ [ module_id = contrail-dns instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.30:8086, ] ] status = Initializing description = Connect : EvTcpConnected ], [ type = Database name = Cassandra server_addrs= [ [ (*_iter6) = 10.0.0.17:9041, (*_iter6) = 10.0.0.12:9041, (*_iter6) = 10.0.0.30:9041, ] ] status = Up description = Established Cassandra connection ], [ type = Database name = RabbitMQ server_addrs= [ [ (*_iter6) = 10.0.0.17:5673, (*_iter6) = 10.0.0.12:5673, (*_iter6) = 10.0.0.30:5673, ] ] status = Up description = RabbitMQ connection established ], ] ] flag_infos= [ [ ] ] description = Collector connection down ], ] ] ] 2025-02-10 Mon 02:00:46:458.760 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604327573248, Pid 78556]: Received UVE message in wrong state : NodeStatusUVE 2025-02-10 Mon 02:00:46:458.769 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604327573248, Pid 78556]: Processing scm::EvSandeshSend in state ClientInit 2025-02-10 Mon 02:00:46:458.775 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604327573248, Pid 78556]: ClientInit : EvSandeshSend : SandeshCtrlClientToServer 2025-02-10 Mon 02:00:46:458.786 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604327573248, Pid 78556]: Processing scm::EvSandeshSend in state ClientInit 2025-02-10 Mon 02:00:46:458.793 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604327573248, Pid 78556]: ClientInit : EvSandeshSend : SandeshModuleClientTrace 2025-02-10 Mon 02:00:46:458.996 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604327573248, Pid 78556]: SANDESH: Send FAILED: 1739152846458702 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-2705-3:Control:contrail-dns:0 client_info= [ status = ClientInit successful_connections = 6 pid = 78556 http_port = 8092 start_time = 1739152566176088 collector_name = collector_ip = 10.0.0.30:8086 collector_list= [ [ (*_iter40) = 10.0.0.17:8086, (*_iter40) = 10.0.0.30:8086, (*_iter40) = 10.0.0.12:8086, ] ] rx_socket_stats= [ bytes = 2034 calls = 4 average_bytes = 508 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 213437 calls = 19 average_bytes = 11233 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ConfigClientGetRowError [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 6 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigClientMgrDebug [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 735 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigurationMalformedReferenceWarningLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 17 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = DnsOperational [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 14 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapLinkOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 469 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapNodeOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 271 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 6 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 32 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlClientToServer [ messages_sent = 5 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlServerToClient [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 10 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 48 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 25 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveDnsInfo [ messages_sent = 2 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 1539, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 80, _iter59->first = sent _iter59->second = 23, ] ] ] 2025-02-10 Mon 02:00:46:459.014 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604327573248, Pid 78556]: Received UVE message in wrong state : SandeshModuleClientTrace 2025-02-10 Mon 02:00:46:478.718 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604570863360, Pid 78556]: OnMessage control in state: ClientInit 2025-02-10 Mon 02:00:46:478.773 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604570863360, Pid 78556]: Processing scm::EvSandeshMessageRecv in state ClientInit 2025-02-10 Mon 02:00:46:478.784 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604570863360, Pid 78556]: ClientInit : EvSandeshMessageRecv 2025-02-10 Mon 02:00:46:478.807 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604570863360, Pid 78556]: Received Ctrl Message with size 0 2025-02-10 Mon 02:00:46:478.832 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604570863360, Pid 78556]: SyncUVE Syncing ObjectDns val name = cn-jenkins-deploy-platform-ansible-os-2705-3 start_time = 1739152566954394 build_info = {"build-info":[{"build-version":"nightly.r24.1.centos","build-time":"2025-02-06 00:13:12.562772","build-user":"root","build-hostname":"jenkins-build-prepare-sandbox-7691-1.novalocal","build-id":"nightly.r24.1.centos-nightly_r24.1_centos.el7@contrail","build-number":"unknown"}]} collector = 10.0.0.17:8086 self_ip_list= [ [ (*_iter12) = 10.20.0.8, ] ] proxy seq 1 2025-02-10 Mon 02:00:46:478.846 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604570863360, Pid 78556]: SyncAllMaps for DnsState:0 period 0 cycle 0 without seqno , total = 1 2025-02-10 Mon 02:00:46:479.127 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604570863360, Pid 78556]: SyncUVE Syncing ObjectGeneratorInfo val name = cn-jenkins-deploy-platform-ansible-os-2705-3:Control:contrail-dns:0 client_info= [ status = ClientInit successful_connections = 6 pid = 78556 http_port = 8092 start_time = 1739152566176088 collector_name = collector_ip = 10.0.0.30:8086 collector_list= [ [ (*_iter40) = 10.0.0.17:8086, (*_iter40) = 10.0.0.30:8086, (*_iter40) = 10.0.0.12:8086, ] ] rx_socket_stats= [ bytes = 2034 calls = 4 average_bytes = 508 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 213437 calls = 19 average_bytes = 11233 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] sm_queue_count = 0 sm_stats= [ ev_stats= [ [ [ event = scm::EvCollectorUpdate enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvConnectTimerExpired enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvIdleHoldTimerExpired enqueues = 7 dequeues = 7 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvSandeshMessageRecv enqueues = 2 dequeues = 2 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvSandeshSend enqueues = 99 dequeues = 99 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvStart enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpClose enqueues = 4 dequeues = 4 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpConnectFail enqueues = 5 dequeues = 5 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpConnected enqueues = 5 dequeues = 5 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpDeleteSession enqueues = 10 dequeues = 10 enqueue_fails = 0 dequeue_fails = 0 ], [ event = enqueues = 135 dequeues = 135 enqueue_fails = 0 dequeue_fails = 0 ], ] ] state = Idle state_since = 1739152816452841 last_state = Idle last_event = scm::EvSandeshSend last_event_at = 1739152816453426 ] session_stats= [ num_recv_msg = 0 num_recv_msg_fail = 0 num_recv_fail = 0 num_send_msg = 1 num_send_msg_fail = 0 num_send_buffer_fail = 0 num_wait_msgq_enqueue = 0 num_wait_msgq_dequeue = 0 num_write_ready_cb_error = 0 ] session_rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] session_tx_socket_stats= [ bytes = 1610 calls = 1 average_bytes = 1610 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] msg_type_agg= [ [ _iter58->first = ConfigClientGetRowError [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 6 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigClientMgrDebug [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 735 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigurationMalformedReferenceWarningLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 17 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = DnsOperational [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 14 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapLinkOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 469 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapNodeOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 271 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 6 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 32 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlClientToServer [ messages_sent = 5 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlServerToClient [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 10 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 48 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 25 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveDnsInfo [ messages_sent = 2 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 1539, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 80, _iter59->first = sent _iter59->second = 23, ] ] max_sm_queue_count = 6 proxy seq 58 2025-02-10 Mon 02:00:46:479.162 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604570863360, Pid 78556]: SyncAllMaps for ModuleClientState:0 period 0 cycle 0 without seqno , total = 1 2025-02-10 Mon 02:00:46:479.188 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604570863360, Pid 78556]: SyncUVE Syncing ObjectDns val name = cn-jenkins-deploy-platform-ansible-os-2705-3 process_status= [ [ [ module_id = contrail-dns instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.30:8086, ] ] status = Initializing description = Connect : EvTcpConnected ], [ type = Database name = Cassandra server_addrs= [ [ (*_iter6) = 10.0.0.17:9041, (*_iter6) = 10.0.0.12:9041, (*_iter6) = 10.0.0.30:9041, ] ] status = Up description = Established Cassandra connection ], [ type = Database name = RabbitMQ server_addrs= [ [ (*_iter6) = 10.0.0.17:5673, (*_iter6) = 10.0.0.12:5673, (*_iter6) = 10.0.0.30:5673, ] ] status = Up description = RabbitMQ connection established ], ] ] flag_infos= [ [ ] ] description = Collector connection down ], ] ] proxy seq 37 2025-02-10 Mon 02:00:46:479.204 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604570863360, Pid 78556]: SyncAllMaps for NodeStatus:0 period 0 cycle 0 without seqno , total = 1 2025-02-10 Mon 02:00:46:479.211 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604570863360, Pid 78556]: SyncAllMaps for PeerFlapData:0 period 0 cycle 0 without seqno , total = 0 2025-02-10 Mon 02:00:46:479.218 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604570863360, Pid 78556]: SyncAllMaps for PeerStatsData:0 period 0 cycle 0 without seqno , total = 0 2025-02-10 Mon 02:00:46:479.225 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604570863360, Pid 78556]: SyncAllMaps for XmppPeerInfoData:0 period 0 cycle 0 without seqno , total = 0 2025-02-10 Mon 02:00:46:479.259 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604570863360, Pid 78556]: Established 2025-02-10 Mon 02:00:50:729.401 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604340168448, Pid 78556]: OnMessage control in state: Established 2025-02-10 Mon 02:00:50:729.458 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604340168448, Pid 78556]: Processing scm::EvSandeshMessageRecv in state Established 2025-02-10 Mon 02:00:50:729.468 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604340168448, Pid 78556]: Established : EvSandeshMessageRecv 2025-02-10 Mon 02:00:50:729.511 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604340168448, Pid 78556]: Received Ctrl Message with size 3 2025-02-10 Mon 02:00:50:729.530 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604340168448, Pid 78556]: SyncAllMaps for DnsState:0 period 0 cycle 0 with seqno 1, total = 0 2025-02-10 Mon 02:00:50:729.542 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604340168448, Pid 78556]: SyncAllMaps for ModuleClientState:0 period 0 cycle 0 with seqno 59, total = 0 2025-02-10 Mon 02:00:50:729.553 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604340168448, Pid 78556]: SyncAllMaps for NodeStatus:0 period 0 cycle 0 with seqno 39, total = 0 2025-02-10 Mon 02:00:50:729.561 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604340168448, Pid 78556]: SyncAllMaps for PeerFlapData:0 period 0 cycle 0 without seqno , total = 0 2025-02-10 Mon 02:00:50:729.567 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604340168448, Pid 78556]: SyncAllMaps for PeerStatsData:0 period 0 cycle 0 without seqno , total = 0 2025-02-10 Mon 02:00:50:729.574 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604340168448, Pid 78556]: SyncAllMaps for XmppPeerInfoData:0 period 0 cycle 0 without seqno , total = 0 2025-02-10 Mon 02:01:05:967.240 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604735187072, Pid 78556]: XMPP [SYS_DEBUG]: XmppCreateConnection: XMPP Peer cn-jenkins-deploy-platform-ansible-os-2705-3:10.20.0.21 SEND Xmpp creating dynamic channel 10.20.0.8:8093::10.20.0.21:43393 controller/src/xmpp/xmpp_server.cc 586 2025-02-10 Mon 02:01:05:967.358 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604735187072, Pid 78556]: XMPP [SYS_INFO]: XmppConnectionCreate: XMPP Peer cn-jenkins-deploy-platform-ansible-os-2705-3:10.20.0.21 RECV Created Xmpp Server connection from cn-jenkins-deploy-platform-ansible-os-2705-3 To : controller/src/xmpp/xmpp_connection.cc 727 2025-02-10 Mon 02:01:05:967.414 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604735187072, Pid 78556]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.20.0.8:8093::10.20.0.21:43393 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 254 2025-02-10 Mon 02:01:05:967.514 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604575061760, Pid 78556]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer cn-jenkins-deploy-platform-ansible-os-2705-3:10.20.0.21 RECV Mode Server: PassiveOpen in state: Idle peer ip: 10.20.0.21 ( ) controller/src/xmpp/xmpp_state_machine.cc 1326 2025-02-10 Mon 02:01:05:967.538 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604331771648, Pid 78556]: [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.21 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1710 2025-02-10 Mon 02:01:05:967.834 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604331771648, Pid 78556]: [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.21 state = Active event = EvTcpPassiveOpen local_port = 8093 remote_port = 43393 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1710 2025-02-10 Mon 02:01:05:967.917 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604331771648, Pid 78556]: [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.21 state = Active next_state = Established event = EvXmppOpen identifier = an-jenkins-deploy-platform-ansible-os-2705-1/dns ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1710 2025-02-10 Mon 02:01:06:059.566 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604735187072, Pid 78556]: XMPP [SYS_DEBUG]: XmppCreateConnection: XMPP Peer cn-jenkins-deploy-platform-ansible-os-2705-3:10.20.0.7 SEND Xmpp creating dynamic channel 10.20.0.8:8093::10.20.0.7:39579 controller/src/xmpp/xmpp_server.cc 586 2025-02-10 Mon 02:01:06:059.638 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604735187072, Pid 78556]: XMPP [SYS_INFO]: XmppConnectionCreate: XMPP Peer cn-jenkins-deploy-platform-ansible-os-2705-3:10.20.0.7 RECV Created Xmpp Server connection from cn-jenkins-deploy-platform-ansible-os-2705-3 To : controller/src/xmpp/xmpp_connection.cc 727 2025-02-10 Mon 02:01:06:059.679 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604735187072, Pid 78556]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.20.0.8:8093::10.20.0.7:39579 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 254 2025-02-10 Mon 02:01:06:059.786 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604335970048, Pid 78556]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer cn-jenkins-deploy-platform-ansible-os-2705-3:10.20.0.7 RECV Mode Server: PassiveOpen in state: Idle peer ip: 10.20.0.7 ( ) controller/src/xmpp/xmpp_state_machine.cc 1326 2025-02-10 Mon 02:01:06:060.118 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604575061760, Pid 78556]: [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.7 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1710 2025-02-10 Mon 02:01:06:060.205 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604575061760, Pid 78556]: [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.7 state = Active event = EvTcpPassiveOpen local_port = 8093 remote_port = 39579 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1710 2025-02-10 Mon 02:01:06:061.123 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604323374848, Pid 78556]: [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.7 state = Active next_state = Established event = EvXmppOpen identifier = an-jenkins-deploy-platform-ansible-os-2705-2/dns ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1710 2025-02-10 Mon 02:01:28:861.786 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604323374848, Pid 78556]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 571 2025-02-10 Mon 02:02:09:457.755 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140604344366848, Pid 78556]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 571 2025-02-10 Mon 02:02:49:072.985 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140460034267264, Pid 109271]: DnsAgent [SYS_INFO]: VersionInfoLog: {"build-info":[{"build-version":"nightly","build-time":"2025-02-10 00:20:54.297706","build-user":"root","build-hostname":"jenkins-build-prepare-sandbox-7708-1.novalocal","build-id":"unknown","build-number":"unknown"}]} src/contrail-common/base/misc_utils.cc 41 2025-02-10 Mon 02:02:49:073.069 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140460034267264, Pid 109271]: SANDESH: ROLE : Generator 2025-02-10 Mon 02:02:49:073.078 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140460034267264, Pid 109271]: SANDESH: MODULE : contrail-dns 2025-02-10 Mon 02:02:49:073.084 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140460034267264, Pid 109271]: SANDESH: SOURCE : cn-jenkins-deploy-platform-ansible-os-2705-3 2025-02-10 Mon 02:02:49:073.090 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140460034267264, Pid 109271]: SANDESH: NODE TYPE : Control 2025-02-10 Mon 02:02:49:073.095 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140460034267264, Pid 109271]: SANDESH: INSTANCE ID : 0 2025-02-10 Mon 02:02:49:073.101 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140460034267264, Pid 109271]: SANDESH: HTTP SERVER PORT : 8092 2025-02-10 Mon 02:02:49:073.107 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140460034267264, Pid 109271]: SANDESH: System Log Send Rate Limit: 0 -> 0 2025-02-10 Mon 02:02:49:077.384 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140460034267264, Pid 109271]: HTTP Introspect Init 2025-02-10 Mon 02:02:49:077.514 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140460034267264, Pid 109271]: TCP [SYS_DEBUG]: TcpServerMessageLog: Server 0.0.0.0:8092 Initialization complete src/contrail-common/io/tcp_server.cc 137 2025-02-10 Mon 02:02:49:077.577 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140460034267264, Pid 109271]: Sandesh Http Server Port: 8092 2025-02-10 Mon 02:02:49:077.623 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140460034267264, Pid 109271]: SANDESH: NOT Writing http_port 8092TO : /tmp/contrail-dns.109163.http_port 2025-02-10 Mon 02:02:49:077.632 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140460034267264, Pid 109271]: SANDESH: CONNECT TO COLLECTOR: 1 2025-02-10 Mon 02:02:49:077.714 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140460034267264, Pid 109271]: Idle 2025-02-10 Mon 02:02:49:078.706 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140460034267264, Pid 109271]: SANDESH: No Client: 1739152969078605 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-2705-3:Control:contrail-dns:0 client_info= [ status = Idle successful_connections = 0 pid = 109271 http_port = 8092 start_time = 1739152969077724 collector_name = collector_ip = 0.0.0.0:0 collector_list= [ [ ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 0, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 1, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 0, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-02-10 Mon 02:02:49:078.850 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140460034267264, Pid 109271]: SANDESH: Logging: DISABLED -> ENABLED 2025-02-10 Mon 02:02:49:078.876 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140460034267264, Pid 109271]: SANDESH: Logging: LEVEL: [ INVALID ] -> [ SYS_DEBUG ] log4level: [ NOTSET ] -> [ DEBUG ] 2025-02-10 Mon 02:02:49:079.101 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: Processing scm::EvStart in state Idle 2025-02-10 Mon 02:02:49:079.154 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: Disconnect 2025-02-10 Mon 02:02:49:081.226 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: Processing scm::EvCollectorUpdate in state Disconnect 2025-02-10 Mon 02:02:49:081.437 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: Connect : Start Connect timer 10.0.0.17:8086 2025-02-10 Mon 02:02:49:081.509 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: Processing scm::EvSandeshSend in state Connect 2025-02-10 Mon 02:02:49:081.594 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: SANDESH: Send FAILED: 1739152969081202 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-2705-3:Control:contrail-dns:0 client_info= [ status = Disconnect successful_connections = 0 pid = 109271 http_port = 8092 start_time = 1739152969077724 collector_name = collector_ip = 0.0.0.0:0 collector_list= [ [ ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 1, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 0, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-02-10 Mon 02:02:49:081.613 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: Wrong state: Connect for event: EvSandeshSend message: SandeshModuleClientTrace 2025-02-10 Mon 02:02:49:081.623 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: Processing scm::EvSandeshSend in state Connect 2025-02-10 Mon 02:02:49:081.697 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: SANDESH: Send FAILED: 1739152969081302 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-2705-3:Control:contrail-dns:0 client_info= [ status = Disconnect successful_connections = 0 pid = 109271 http_port = 8092 start_time = 1739152969077724 collector_name = collector_ip = 10.0.0.17:8086 collector_list= [ [ (*_iter40) = 10.0.0.17:8086, (*_iter40) = 10.0.0.30:8086, (*_iter40) = 10.0.0.12:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 1, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 0, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-02-10 Mon 02:02:49:081.706 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: Wrong state: Connect for event: EvSandeshSend message: SandeshModuleClientTrace 2025-02-10 Mon 02:02:49:081.718 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: Processing scm::EvSandeshSend in state Connect 2025-02-10 Mon 02:02:49:081.780 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: SANDESH: Send FAILED: 1739152969081500 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-2705-3:Control:contrail-dns:0 client_info= [ status = Connect successful_connections = 0 pid = 109271 http_port = 8092 start_time = 1739152969077724 collector_name = collector_ip = 10.0.0.17:8086 collector_list= [ [ (*_iter40) = 10.0.0.17:8086, (*_iter40) = 10.0.0.30:8086, (*_iter40) = 10.0.0.12:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 1, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 0, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-02-10 Mon 02:02:49:081.789 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: Wrong state: Connect for event: EvSandeshSend message: SandeshModuleClientTrace 2025-02-10 Mon 02:02:49:200.618 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140460034267264, Pid 109271]: Applying named configuration failed 2025-02-10 Mon 02:02:49:200.986 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140460034267264, Pid 109271]: TCP [SYS_DEBUG]: TcpServerMessageLog: Server 10.20.0.8:8093 Initialization complete src/contrail-common/io/tcp_server.cc 137 2025-02-10 Mon 02:02:49:201.069 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140460034267264, Pid 109271]: DnsAgent [SYS_NOTICE]: DnsOperational: Bulk sync has been started controller/src/dns/main.cc 384 2025-02-10 Mon 02:02:49:201.143 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459657656064, Pid 109271]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Config Client Mgr SM: Start RabbitMqReader and init Database src/contrail-common/config-client-mgr/config_client_manager.cc 303 2025-02-10 Mon 02:02:49:202.116 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459657656064, Pid 109271]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: RabbitMQ SM: RabbitMQ connection down src/contrail-common/config-client-mgr/config_amqp_client.cc 158 2025-02-10 Mon 02:02:49:202.140 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459657656064, Pid 109271]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: RabbitMQ SM: Connect to Rabbit MQ with queue_delete TRUE src/contrail-common/config-client-mgr/config_amqp_client.cc 167 2025-02-10 Mon 02:02:49:202.330 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459645060864, Pid 109271]: Processing scm::EvSandeshSend in state Connect 2025-02-10 Mon 02:02:49:203.252 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459645060864, Pid 109271]: SANDESH: Send FAILED: 1739152969201243 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-2705-3 process_status= [ [ [ module_id = contrail-dns instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.17:8086, ] ] status = Initializing description = Disconnect : EvCollectorUpdate -> Connect ], [ type = Database name = RabbitMQ server_addrs= [ [ (*_iter6) = 10.0.0.17:5673, (*_iter6) = 10.0.0.12:5673, (*_iter6) = 10.0.0.30:5673, ] ] status = Down description = RabbitMQ connection down ], ] ] flag_infos= [ [ ] ] description = Number of connections:2, Expected:3 Missing: Database:Cassandra ], ] ] ] 2025-02-10 Mon 02:02:49:203.269 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459645060864, Pid 109271]: Wrong state: Connect for event: EvSandeshSend message: NodeStatusUVE 2025-02-10 Mon 02:02:49:270.945 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459657656064, Pid 109271]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: RabbitMQ SM: RabbitMQ connection established src/contrail-common/config-client-mgr/config_amqp_client.cc 151 2025-02-10 Mon 02:02:49:270.978 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459657656064, Pid 109271]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Config Client Mgr SM: Init Database src/contrail-common/config-client-mgr/config_client_manager.cc 307 2025-02-10 Mon 02:02:49:271.016 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459657656064, Pid 109271]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Lost Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 580 2025-02-10 Mon 02:02:49:271.026 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459657656064, Pid 109271]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Db Init src/contrail-common/config-client-mgr/config_cassandra_client.cc 87 2025-02-10 Mon 02:02:49:272.542 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: Processing scm::EvSandeshSend in state Connect 2025-02-10 Mon 02:02:49:272.591 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: SANDESH: Send FAILED: 1739152969270900 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-2705-3 process_status= [ [ [ module_id = contrail-dns instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.17:8086, ] ] status = Initializing description = Disconnect : EvCollectorUpdate -> Connect ], [ type = Database name = RabbitMQ server_addrs= [ [ (*_iter6) = 10.0.0.17:5673, (*_iter6) = 10.0.0.12:5673, (*_iter6) = 10.0.0.30:5673, ] ] status = Up description = RabbitMQ connection established ], ] ] flag_infos= [ [ ] ] description = Number of connections:2, Expected:3 Missing: Database:Cassandra ], ] ] ] 2025-02-10 Mon 02:02:49:272.607 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: Wrong state: Connect for event: EvSandeshSend message: NodeStatusUVE 2025-02-10 Mon 02:02:49:272.617 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: Processing scm::EvSandeshSend in state Connect 2025-02-10 Mon 02:02:49:272.637 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: SANDESH: Send FAILED: 1739152969271009 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-2705-3 process_status= [ [ [ module_id = contrail-dns instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.17:8086, ] ] status = Initializing description = Disconnect : EvCollectorUpdate -> Connect ], [ type = Database name = Cassandra server_addrs= [ [ (*_iter6) = 10.0.0.17:9041, (*_iter6) = 10.0.0.12:9041, (*_iter6) = 10.0.0.30:9041, ] ] status = Down description = Lost Cassandra connection ], [ type = Database name = RabbitMQ server_addrs= [ [ (*_iter6) = 10.0.0.17:5673, (*_iter6) = 10.0.0.12:5673, (*_iter6) = 10.0.0.30:5673, ] ] status = Up description = RabbitMQ connection established ], ] ] flag_infos= [ [ ] ] description = Collector, Database:Cassandra connection down ], ] ] ] 2025-02-10 Mon 02:02:49:272.644 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: Wrong state: Connect for event: EvSandeshSend message: NodeStatusUVE 2025-02-10 Mon 02:02:49:346.013 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459657656064, Pid 109271]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Db SetTableSpace src/contrail-common/config-client-mgr/config_cassandra_client.cc 95 2025-02-10 Mon 02:02:49:351.340 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459657656064, Pid 109271]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Db UseColumnFamily uuidTableName src/contrail-common/config-client-mgr/config_cassandra_client.cc 104 2025-02-10 Mon 02:02:49:351.387 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459657656064, Pid 109271]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Db UseColumnFamily fqnTableName src/contrail-common/config-client-mgr/config_cassandra_client.cc 110 2025-02-10 Mon 02:02:49:351.460 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459657656064, Pid 109271]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-02-10 Mon 02:02:49:351.471 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459657656064, Pid 109271]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: BulkDataSync Started src/contrail-common/config-client-mgr/config_cassandra_client.cc 393 2025-02-10 Mon 02:02:49:351.537 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: Processing scm::EvSandeshSend in state Connect 2025-02-10 Mon 02:02:49:351.575 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: SANDESH: Send FAILED: 1739152969351432 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-2705-3 process_status= [ [ [ module_id = contrail-dns instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.17:8086, ] ] status = Initializing description = Disconnect : EvCollectorUpdate -> Connect ], [ type = Database name = Cassandra server_addrs= [ [ (*_iter6) = 10.0.0.17:9041, (*_iter6) = 10.0.0.12:9041, (*_iter6) = 10.0.0.30:9041, ] ] status = Up description = Established Cassandra connection ], [ type = Database name = RabbitMQ server_addrs= [ [ (*_iter6) = 10.0.0.17:5673, (*_iter6) = 10.0.0.12:5673, (*_iter6) = 10.0.0.30:5673, ] ] status = Up description = RabbitMQ connection established ], ] ] flag_infos= [ [ ] ] description = Collector connection down ], ] ] ] 2025-02-10 Mon 02:02:49:351.594 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: Wrong state: Connect for event: EvSandeshSend message: NodeStatusUVE 2025-02-10 Mon 02:02:49:354.533 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459632465664, Pid 109271]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-02-10 Mon 02:02:49:360.562 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459632465664, Pid 109271]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-02-10 Mon 02:02:49:362.709 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459632465664, Pid 109271]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-02-10 Mon 02:02:49:363.983 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459632465664, Pid 109271]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-02-10 Mon 02:02:49:365.253 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459632465664, Pid 109271]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-02-10 Mon 02:02:49:366.034 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459632465664, Pid 109271]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-02-10 Mon 02:02:49:374.914 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459632465664, Pid 109271]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-02-10 Mon 02:02:49:376.676 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459632465664, Pid 109271]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-02-10 Mon 02:02:49:383.745 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459632465664, Pid 109271]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-02-10 Mon 02:02:49:385.110 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459632465664, Pid 109271]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-02-10 Mon 02:02:49:389.220 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459632465664, Pid 109271]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-02-10 Mon 02:02:49:402.428 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459632465664, Pid 109271]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-02-10 Mon 02:02:49:407.631 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459632465664, Pid 109271]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-02-10 Mon 02:02:49:418.774 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459632465664, Pid 109271]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-02-10 Mon 02:02:49:427.622 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459632465664, Pid 109271]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-02-10 Mon 02:02:49:429.021 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459632465664, Pid 109271]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-02-10 Mon 02:02:49:430.351 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459632465664, Pid 109271]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-02-10 Mon 02:02:49:435.416 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459632465664, Pid 109271]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-02-10 Mon 02:02:49:438.451 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459632465664, Pid 109271]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-02-10 Mon 02:02:49:443.415 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459632465664, Pid 109271]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-02-10 Mon 02:02:49:444.299 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459632465664, Pid 109271]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-02-10 Mon 02:02:49:445.396 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459632465664, Pid 109271]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-02-10 Mon 02:02:49:446.264 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459632465664, Pid 109271]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-02-10 Mon 02:02:49:448.007 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459632465664, Pid 109271]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-02-10 Mon 02:02:49:449.575 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459632465664, Pid 109271]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-02-10 Mon 02:02:49:452.531 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459632465664, Pid 109271]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-02-10 Mon 02:02:49:454.105 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459632465664, Pid 109271]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-02-10 Mon 02:02:49:454.948 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459632465664, Pid 109271]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-02-10 Mon 02:02:49:456.250 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459632465664, Pid 109271]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-02-10 Mon 02:02:49:458.377 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459632465664, Pid 109271]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-02-10 Mon 02:02:49:459.269 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459632465664, Pid 109271]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-02-10 Mon 02:02:49:460.340 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459632465664, Pid 109271]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-02-10 Mon 02:02:49:462.538 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459632465664, Pid 109271]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-02-10 Mon 02:02:49:467.258 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459632465664, Pid 109271]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-02-10 Mon 02:02:49:468.678 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459632465664, Pid 109271]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-02-10 Mon 02:02:49:471.077 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459632465664, Pid 109271]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-02-10 Mon 02:02:49:474.287 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459632465664, Pid 109271]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-02-10 Mon 02:02:49:476.860 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459632465664, Pid 109271]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-02-10 Mon 02:02:49:482.306 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459632465664, Pid 109271]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-02-10 Mon 02:02:49:489.292 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459632465664, Pid 109271]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-02-10 Mon 02:02:49:490.367 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459632465664, Pid 109271]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-02-10 Mon 02:02:49:501.677 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459632465664, Pid 109271]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-02-10 Mon 02:02:49:503.477 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459632465664, Pid 109271]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-02-10 Mon 02:02:49:505.259 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459632465664, Pid 109271]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-02-10 Mon 02:02:49:527.259 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459632465664, Pid 109271]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-02-10 Mon 02:02:49:529.717 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459632465664, Pid 109271]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-02-10 Mon 02:02:49:531.701 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459632465664, Pid 109271]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-02-10 Mon 02:02:49:534.503 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459632465664, Pid 109271]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-02-10 Mon 02:02:49:535.464 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459632465664, Pid 109271]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-02-10 Mon 02:02:49:536.443 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459632465664, Pid 109271]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-02-10 Mon 02:02:49:538.373 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459632465664, Pid 109271]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-02-10 Mon 02:02:49:539.377 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459632465664, Pid 109271]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-02-10 Mon 02:02:49:540.859 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459632465664, Pid 109271]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-02-10 Mon 02:02:49:547.789 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459632465664, Pid 109271]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-02-10 Mon 02:02:49:548.864 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459632465664, Pid 109271]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-02-10 Mon 02:02:49:549.669 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459632465664, Pid 109271]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-02-10 Mon 02:02:49:551.476 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459632465664, Pid 109271]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-02-10 Mon 02:02:49:552.478 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459632465664, Pid 109271]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-02-10 Mon 02:02:49:553.345 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459632465664, Pid 109271]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-02-10 Mon 02:02:49:554.242 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459632465664, Pid 109271]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-02-10 Mon 02:02:49:555.026 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459632465664, Pid 109271]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-02-10 Mon 02:02:49:555.773 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459632465664, Pid 109271]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-02-10 Mon 02:02:49:557.357 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459632465664, Pid 109271]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-02-10 Mon 02:02:49:558.752 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459632465664, Pid 109271]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-02-10 Mon 02:02:49:559.553 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459632465664, Pid 109271]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-02-10 Mon 02:02:49:560.926 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459632465664, Pid 109271]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-02-10 Mon 02:02:49:561.936 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459632465664, Pid 109271]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-02-10 Mon 02:02:49:562.658 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459632465664, Pid 109271]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-02-10 Mon 02:02:49:563.631 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459632465664, Pid 109271]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-02-10 Mon 02:02:49:565.205 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459632465664, Pid 109271]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-02-10 Mon 02:02:49:565.993 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459632465664, Pid 109271]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-02-10 Mon 02:02:49:571.883 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459632465664, Pid 109271]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-02-10 Mon 02:02:49:573.269 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459632465664, Pid 109271]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-02-10 Mon 02:02:49:574.277 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459632465664, Pid 109271]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-02-10 Mon 02:02:49:576.154 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459632465664, Pid 109271]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-02-10 Mon 02:02:49:578.624 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459632465664, Pid 109271]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-02-10 Mon 02:02:49:579.563 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459632465664, Pid 109271]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-02-10 Mon 02:02:49:581.348 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459632465664, Pid 109271]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-02-10 Mon 02:02:49:582.218 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459632465664, Pid 109271]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-02-10 Mon 02:02:49:583.989 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459632465664, Pid 109271]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-02-10 Mon 02:02:49:589.322 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459632465664, Pid 109271]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-02-10 Mon 02:02:49:590.404 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459632465664, Pid 109271]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-02-10 Mon 02:02:49:591.366 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459632465664, Pid 109271]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-02-10 Mon 02:02:49:600.253 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459632465664, Pid 109271]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-02-10 Mon 02:02:49:602.421 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459632465664, Pid 109271]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-02-10 Mon 02:02:49:603.440 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459632465664, Pid 109271]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-02-10 Mon 02:02:49:611.402 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459632465664, Pid 109271]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-02-10 Mon 02:02:49:613.140 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459632465664, Pid 109271]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-02-10 Mon 02:02:49:614.745 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459632465664, Pid 109271]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-02-10 Mon 02:02:49:615.669 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459632465664, Pid 109271]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-02-10 Mon 02:02:49:616.980 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459632465664, Pid 109271]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-02-10 Mon 02:02:49:617.810 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459632465664, Pid 109271]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-02-10 Mon 02:02:49:621.523 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459632465664, Pid 109271]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-02-10 Mon 02:02:49:623.078 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459632465664, Pid 109271]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-02-10 Mon 02:02:49:623.991 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459632465664, Pid 109271]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-02-10 Mon 02:02:49:624.741 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459632465664, Pid 109271]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-02-10 Mon 02:02:49:628.678 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459632465664, Pid 109271]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-02-10 Mon 02:02:49:629.762 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459632465664, Pid 109271]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-02-10 Mon 02:02:49:630.544 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459632465664, Pid 109271]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-02-10 Mon 02:02:49:631.424 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459632465664, Pid 109271]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-02-10 Mon 02:02:49:632.755 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459632465664, Pid 109271]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-02-10 Mon 02:02:49:634.063 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459632465664, Pid 109271]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-02-10 Mon 02:02:49:635.345 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459632465664, Pid 109271]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-02-10 Mon 02:02:49:636.067 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459632465664, Pid 109271]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-02-10 Mon 02:02:49:636.767 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459632465664, Pid 109271]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-02-10 Mon 02:02:49:637.587 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459632465664, Pid 109271]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-02-10 Mon 02:02:49:639.008 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459632465664, Pid 109271]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-02-10 Mon 02:02:49:640.274 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459632465664, Pid 109271]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-02-10 Mon 02:02:49:641.683 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459632465664, Pid 109271]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-02-10 Mon 02:02:49:642.475 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459632465664, Pid 109271]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-02-10 Mon 02:02:49:643.236 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459632465664, Pid 109271]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-02-10 Mon 02:02:49:644.083 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459632465664, Pid 109271]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-02-10 Mon 02:02:49:644.802 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459632465664, Pid 109271]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-02-10 Mon 02:02:49:650.281 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459632465664, Pid 109271]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-02-10 Mon 02:02:49:652.048 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459632465664, Pid 109271]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-02-10 Mon 02:02:49:652.907 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459632465664, Pid 109271]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-02-10 Mon 02:02:49:654.854 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459632465664, Pid 109271]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-02-10 Mon 02:02:49:655.810 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459632465664, Pid 109271]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-02-10 Mon 02:02:49:655.857 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459632465664, Pid 109271]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: One reader finished BulkSync src/contrail-common/config-client-mgr/config_cassandra_client.cc 558 2025-02-10 Mon 02:02:49:655.888 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459636664064, Pid 109271]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: One reader finished BulkSync src/contrail-common/config-client-mgr/config_cassandra_client.cc 558 2025-02-10 Mon 02:02:49:656.112 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459645060864, Pid 109271]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: One reader finished BulkSync src/contrail-common/config-client-mgr/config_cassandra_client.cc 558 2025-02-10 Mon 02:02:49:656.274 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459867805440, Pid 109271]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: One reader finished BulkSync src/contrail-common/config-client-mgr/config_cassandra_client.cc 558 2025-02-10 Mon 02:02:49:656.431 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459640862464, Pid 109271]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: One reader finished BulkSync src/contrail-common/config-client-mgr/config_cassandra_client.cc 558 2025-02-10 Mon 02:02:49:656.546 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: One reader finished BulkSync src/contrail-common/config-client-mgr/config_cassandra_client.cc 558 2025-02-10 Mon 02:02:49:656.711 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459649259264, Pid 109271]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: One reader finished BulkSync src/contrail-common/config-client-mgr/config_cassandra_client.cc 558 2025-02-10 Mon 02:02:49:656.834 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459872003840, Pid 109271]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: BulkSyncDone by all readers src/contrail-common/config-client-mgr/config_cassandra_client.cc 554 2025-02-10 Mon 02:02:49:656.886 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459872003840, Pid 109271]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Config Client Mgr SM: End of RIB computed and notification sent src/contrail-common/config-client-mgr/config_client_manager.cc 197 2025-02-10 Mon 02:02:49:656.901 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459657656064, Pid 109271]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: TRUE src/contrail-common/config-client-mgr/config_client_manager.cc 225 2025-02-10 Mon 02:02:49:661.370 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459657656064, Pid 109271]: Processing scm::EvSandeshSend in state Connect 2025-02-10 Mon 02:02:49:661.431 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459657656064, Pid 109271]: SANDESH: Send FAILED: 1739152969656959 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-2705-3 process_status= [ [ [ module_id = contrail-dns instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.17:8086, ] ] status = Initializing description = Disconnect : EvCollectorUpdate -> Connect ], [ type = Database name = Cassandra server_addrs= [ [ (*_iter6) = 10.0.0.17:9041, (*_iter6) = 10.0.0.12:9041, (*_iter6) = 10.0.0.30:9041, ] ] status = Up description = Established Cassandra connection ], [ type = Database name = RabbitMQ server_addrs= [ [ (*_iter6) = 10.0.0.17:5673, (*_iter6) = 10.0.0.12:5673, (*_iter6) = 10.0.0.30:5673, ] ] status = Up description = RabbitMQ connection established ], ] ] flag_infos= [ [ ] ] description = Collector connection down ], ] ] ] 2025-02-10 Mon 02:02:49:661.441 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459657656064, Pid 109271]: Wrong state: Connect for event: EvSandeshSend message: NodeStatusUVE 2025-02-10 Mon 02:02:49:666.387 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459632465664, Pid 109271]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-02-10 Mon 02:02:49:667.182 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459632465664, Pid 109271]: IFMap [SYS_WARN]: ConfigurationMalformedReferenceWarningLog: bgp_peering No entry in metadata map bgp_router 9ace496e-24e3-4a41-918e-2a6e3adb9920 controller/src/ifmap/client/config_json_parser.cc 223 2025-02-10 Mon 02:02:49:667.206 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459632465664, Pid 109271]: IFMap [SYS_WARN]: ConfigurationMalformedReferenceWarningLog: bgp_peering No entry in metadata map bgp_router 9ace496e-24e3-4a41-918e-2a6e3adb9920 controller/src/ifmap/client/config_json_parser.cc 223 2025-02-10 Mon 02:02:49:667.693 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459632465664, Pid 109271]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating config-database-node:default-global-system-config:cn-jenkins-deploy-platform-ansible-os-2705-1. controller/src/ifmap/ifmap_server_table.cc 95 2025-02-10 Mon 02:02:49:667.724 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459632465664, Pid 109271]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating global-system-config:default-global-system-config controller/src/ifmap/ifmap_server_table.cc 95 2025-02-10 Mon 02:02:49:667.740 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459632465664, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-config-database-node controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:667.754 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459632465664, Pid 109271]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating role-config:default-global-system-config:juniper-qfx5k-lean:basic controller/src/ifmap/ifmap_server_table.cc 95 2025-02-10 Mon 02:02:49:667.786 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459632465664, Pid 109271]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating node-profile:default-global-system-config:juniper-qfx5k-lean controller/src/ifmap/ifmap_server_table.cc 95 2025-02-10 Mon 02:02:49:667.798 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459632465664, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating node-profile-role-config controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:667.811 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459632465664, Pid 109271]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating device-functional-group:default-domain:default-project:L3-Server-Leaf controller/src/ifmap/ifmap_server_table.cc 95 2025-02-10 Mon 02:02:49:667.826 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459632465664, Pid 109271]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating project:default-domain:default-project controller/src/ifmap/ifmap_server_table.cc 95 2025-02-10 Mon 02:02:49:667.837 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459632465664, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating project-device-functional-group controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:667.850 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459632465664, Pid 109271]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating physical-role:default-global-system-config:leaf controller/src/ifmap/ifmap_server_table.cc 95 2025-02-10 Mon 02:02:49:667.861 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459632465664, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating device-functional-group-physical-role controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:667.885 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459632465664, Pid 109271]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating telemetry-profile:default-domain:default-project:default-telemetry-profile-1 controller/src/ifmap/ifmap_server_table.cc 95 2025-02-10 Mon 02:02:49:667.900 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459632465664, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating project-telemetry-profile controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:667.914 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459632465664, Pid 109271]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating sflow-profile:default-domain:default-project:sflow-all-interfaces controller/src/ifmap/ifmap_server_table.cc 95 2025-02-10 Mon 02:02:49:667.924 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459632465664, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating telemetry-profile-sflow-profile controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:667.936 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459632465664, Pid 109271]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating discovery-service-assignment:default-discovery-service-assignment controller/src/ifmap/ifmap_server_table.cc 95 2025-02-10 Mon 02:02:49:667.946 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459632465664, Pid 109271]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating tag-type:neutron_fwaas controller/src/ifmap/ifmap_server_table.cc 95 2025-02-10 Mon 02:02:49:667.957 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459632465664, Pid 109271]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating hardware:juniper-qfx5210-64c controller/src/ifmap/ifmap_server_table.cc 95 2025-02-10 Mon 02:02:49:667.968 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459632465664, Pid 109271]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating hardware:juniper-qfx5100-96s-8q controller/src/ifmap/ifmap_server_table.cc 95 2025-02-10 Mon 02:02:49:667.979 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459632465664, Pid 109271]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating job-template:default-global-system-config:discover_role_template controller/src/ifmap/ifmap_server_table.cc 95 2025-02-10 Mon 02:02:49:668.006 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459632465664, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-job-template controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:668.019 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459632465664, Pid 109271]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating hardware:juniper-srx5600 controller/src/ifmap/ifmap_server_table.cc 95 2025-02-10 Mon 02:02:49:668.029 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459632465664, Pid 109271]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating hardware:juniper-srx5400 controller/src/ifmap/ifmap_server_table.cc 95 2025-02-10 Mon 02:02:49:668.039 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459632465664, Pid 109271]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating job-template:default-global-system-config:topology_discovery_template controller/src/ifmap/ifmap_server_table.cc 95 2025-02-10 Mon 02:02:49:668.057 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459632465664, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-job-template controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:668.070 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459632465664, Pid 109271]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating access-control-list:default-domain:default-project:__no_rule__:egress-access-control-list controller/src/ifmap/ifmap_server_table.cc 95 2025-02-10 Mon 02:02:49:668.091 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459632465664, Pid 109271]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating security-group:default-domain:default-project:__no_rule__ controller/src/ifmap/ifmap_server_table.cc 95 2025-02-10 Mon 02:02:49:668.102 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459632465664, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating security-group-access-control-list controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:668.114 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459632465664, Pid 109271]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating hardware:juniper-vsrx controller/src/ifmap/ifmap_server_table.cc 95 2025-02-10 Mon 02:02:49:668.125 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459632465664, Pid 109271]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating overlay-role:default-global-system-config:pnf-servicechain controller/src/ifmap/ifmap_server_table.cc 95 2025-02-10 Mon 02:02:49:668.139 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459632465664, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-overlay-role controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:668.151 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459632465664, Pid 109271]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating tag-type:application controller/src/ifmap/ifmap_server_table.cc 95 2025-02-10 Mon 02:02:49:668.162 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459632465664, Pid 109271]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating tag-type:deployment controller/src/ifmap/ifmap_server_table.cc 95 2025-02-10 Mon 02:02:49:668.172 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459632465664, Pid 109271]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating job-template:default-global-system-config:discover_server_template controller/src/ifmap/ifmap_server_table.cc 95 2025-02-10 Mon 02:02:49:668.192 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459632465664, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-job-template controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:668.205 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459632465664, Pid 109271]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating hardware:juniper-qfx5100-24q-aa controller/src/ifmap/ifmap_server_table.cc 95 2025-02-10 Mon 02:02:49:668.217 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459632465664, Pid 109271]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating overlay-role:default-global-system-config:erb-ucast-gateway controller/src/ifmap/ifmap_server_table.cc 95 2025-02-10 Mon 02:02:49:668.231 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459632465664, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-overlay-role controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:668.242 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459632465664, Pid 109271]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating virtual-network:default-domain:default-project:__link_local__ controller/src/ifmap/ifmap_server_table.cc 95 2025-02-10 Mon 02:02:49:668.272 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459632465664, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating project-virtual-network controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:668.284 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459632465664, Pid 109271]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating device-functional-group:default-domain:default-project:Border-Leaf-in-Edge-Routed controller/src/ifmap/ifmap_server_table.cc 95 2025-02-10 Mon 02:02:49:668.302 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459632465664, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating project-device-functional-group controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:668.318 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459632465664, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating device-functional-group-physical-role controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:668.330 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459632465664, Pid 109271]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating alarm:default-global-system-config:system-defined-pending-cassandra-compaction-tasks controller/src/ifmap/ifmap_server_table.cc 95 2025-02-10 Mon 02:02:49:668.347 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459632465664, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-alarm controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:668.359 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459632465664, Pid 109271]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating hardware:juniper-qfx5120-48y-8c controller/src/ifmap/ifmap_server_table.cc 95 2025-02-10 Mon 02:02:49:668.373 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459632465664, Pid 109271]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating role-definition:default-global-system-config:pnf-servicechain-leaf controller/src/ifmap/ifmap_server_table.cc 95 2025-02-10 Mon 02:02:49:668.388 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459632465664, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-role-definition controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:668.402 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459632465664, Pid 109271]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating feature:default-global-system-config:pnf-service-chaining controller/src/ifmap/ifmap_server_table.cc 95 2025-02-10 Mon 02:02:49:668.412 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459632465664, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:668.427 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459632465664, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-overlay-role controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:668.441 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459632465664, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-physical-role controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:668.458 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459632465664, Pid 109271]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating hardware:juniper-srx1500 controller/src/ifmap/ifmap_server_table.cc 95 2025-02-10 Mon 02:02:49:668.468 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459632465664, Pid 109271]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating hardware:juniper-qfx5100e-96s-8q controller/src/ifmap/ifmap_server_table.cc 95 2025-02-10 Mon 02:02:49:668.479 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459632465664, Pid 109271]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating namespace:default-domain:default-namespace controller/src/ifmap/ifmap_server_table.cc 95 2025-02-10 Mon 02:02:49:668.491 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459632465664, Pid 109271]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating domain:default-domain controller/src/ifmap/ifmap_server_table.cc 95 2025-02-10 Mon 02:02:49:668.502 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459632465664, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating domain-namespace controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:668.513 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459632465664, Pid 109271]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating device-functional-group:default-domain:default-project:L2-Server-Leaf controller/src/ifmap/ifmap_server_table.cc 95 2025-02-10 Mon 02:02:49:668.530 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459632465664, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating project-device-functional-group controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:668.545 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459632465664, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating device-functional-group-physical-role controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:668.561 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459632465664, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-physical-role controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:669.746 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459649259264, Pid 109271]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-02-10 Mon 02:02:49:671.104 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459649259264, Pid 109271]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating alarm:default-global-system-config:system-defined-bgp-connectivity controller/src/ifmap/ifmap_server_table.cc 95 2025-02-10 Mon 02:02:49:671.136 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459649259264, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-alarm controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:671.152 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459649259264, Pid 109271]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating feature:default-global-system-config:telemetry controller/src/ifmap/ifmap_server_table.cc 95 2025-02-10 Mon 02:02:49:671.170 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459649259264, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:671.183 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459649259264, Pid 109271]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating alarm:default-global-system-config:system-defined-vrouter-table-limit-exceeded controller/src/ifmap/ifmap_server_table.cc 95 2025-02-10 Mon 02:02:49:671.199 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459649259264, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-alarm controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:671.219 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459649259264, Pid 109271]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating feature-config:default-global-system-config:ar-client-leaf:assisted-replicator controller/src/ifmap/ifmap_server_table.cc 95 2025-02-10 Mon 02:02:49:671.234 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459649259264, Pid 109271]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating role-definition:default-global-system-config:ar-client-leaf controller/src/ifmap/ifmap_server_table.cc 95 2025-02-10 Mon 02:02:49:671.245 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459649259264, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature-config controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:671.257 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459649259264, Pid 109271]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating feature:default-global-system-config:dc-gateway controller/src/ifmap/ifmap_server_table.cc 95 2025-02-10 Mon 02:02:49:671.271 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459649259264, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:671.284 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459649259264, Pid 109271]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating job-template:default-global-system-config:existing_fabric_onboard_template controller/src/ifmap/ifmap_server_table.cc 95 2025-02-10 Mon 02:02:49:671.304 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459649259264, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-job-template controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:671.319 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459649259264, Pid 109271]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating hardware:juniper-qfx5220-32cd controller/src/ifmap/ifmap_server_table.cc 95 2025-02-10 Mon 02:02:49:671.357 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459649259264, Pid 109271]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating virtual-router:default-global-system-config:an-jenkins-deploy-platform-ansible-os-2705-2 controller/src/ifmap/ifmap_server_table.cc 95 2025-02-10 Mon 02:02:49:671.375 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459649259264, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-virtual-router controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:671.387 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459649259264, Pid 109271]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating feature:default-global-system-config:overlay-bgp controller/src/ifmap/ifmap_server_table.cc 95 2025-02-10 Mon 02:02:49:671.401 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459649259264, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:671.414 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459649259264, Pid 109271]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating hardware:juniper-qfx5200-48y controller/src/ifmap/ifmap_server_table.cc 95 2025-02-10 Mon 02:02:49:671.424 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459649259264, Pid 109271]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating job-template:default-global-system-config:server_import_template controller/src/ifmap/ifmap_server_table.cc 95 2025-02-10 Mon 02:02:49:671.442 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459649259264, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-job-template controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:671.454 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459649259264, Pid 109271]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating analytics-snmp-node:default-global-system-config:cn-jenkins-deploy-platform-ansible-os-2705-2. controller/src/ifmap/ifmap_server_table.cc 95 2025-02-10 Mon 02:02:49:671.472 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459649259264, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-analytics-snmp-node controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:671.485 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459649259264, Pid 109271]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating job-template:default-global-system-config:fabric_deletion_template controller/src/ifmap/ifmap_server_table.cc 95 2025-02-10 Mon 02:02:49:671.504 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459649259264, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-job-template controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:671.516 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459649259264, Pid 109271]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating virtual-network:default-domain:default-project:default-virtual-network controller/src/ifmap/ifmap_server_table.cc 95 2025-02-10 Mon 02:02:49:671.534 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459649259264, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating project-virtual-network controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:671.546 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459649259264, Pid 109271]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating feature:default-global-system-config:l3-gateway controller/src/ifmap/ifmap_server_table.cc 95 2025-02-10 Mon 02:02:49:671.560 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459649259264, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:671.572 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459649259264, Pid 109271]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating database-node:default-global-system-config:cn-jenkins-deploy-platform-ansible-os-2705-2. controller/src/ifmap/ifmap_server_table.cc 95 2025-02-10 Mon 02:02:49:671.586 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459649259264, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-database-node controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:671.598 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459649259264, Pid 109271]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating role-definition:default-global-system-config:collapsed-spine controller/src/ifmap/ifmap_server_table.cc 95 2025-02-10 Mon 02:02:49:671.612 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459649259264, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-role-definition controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:671.627 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459649259264, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:671.640 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459649259264, Pid 109271]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating feature:default-global-system-config:vn-interconnect controller/src/ifmap/ifmap_server_table.cc 95 2025-02-10 Mon 02:02:49:671.650 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459649259264, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:671.664 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459649259264, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:671.683 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459649259264, Pid 109271]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating feature:default-global-system-config:underlay-ip-clos controller/src/ifmap/ifmap_server_table.cc 95 2025-02-10 Mon 02:02:49:671.693 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459649259264, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:671.707 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459649259264, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:671.719 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459649259264, Pid 109271]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating feature:default-global-system-config:l2-gateway controller/src/ifmap/ifmap_server_table.cc 95 2025-02-10 Mon 02:02:49:671.729 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459649259264, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:671.741 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459649259264, Pid 109271]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating feature:default-global-system-config:firewall controller/src/ifmap/ifmap_server_table.cc 95 2025-02-10 Mon 02:02:49:671.751 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459649259264, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:671.763 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459649259264, Pid 109271]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating feature:default-global-system-config:infra-bms-access controller/src/ifmap/ifmap_server_table.cc 95 2025-02-10 Mon 02:02:49:671.773 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459649259264, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:671.785 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459649259264, Pid 109271]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating feature:default-global-system-config:port-profile controller/src/ifmap/ifmap_server_table.cc 95 2025-02-10 Mon 02:02:49:671.795 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459649259264, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:671.808 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459649259264, Pid 109271]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating overlay-role:default-global-system-config:collapsed-spine controller/src/ifmap/ifmap_server_table.cc 95 2025-02-10 Mon 02:02:49:671.818 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459649259264, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-overlay-role controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:671.832 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459649259264, Pid 109271]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating physical-role:default-global-system-config:spine controller/src/ifmap/ifmap_server_table.cc 95 2025-02-10 Mon 02:02:49:671.842 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459649259264, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-physical-role controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:671.854 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459649259264, Pid 109271]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating virtual-network:default-domain:default-project:_internal_vn_ipv6_link_local controller/src/ifmap/ifmap_server_table.cc 95 2025-02-10 Mon 02:02:49:671.885 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459649259264, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating project-virtual-network controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:671.900 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459649259264, Pid 109271]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating network-ipam:default-domain:default-project:_internal_ipam_ipv6_link_local controller/src/ifmap/ifmap_server_table.cc 95 2025-02-10 Mon 02:02:49:671.910 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459649259264, Pid 109271]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating virtual-network-network-ipam:attr(default-domain:default-project:_internal_ipam_ipv6_link_local,default-domain:default-project:_internal_vn_ipv6_link_local) controller/src/ifmap/ifmap_server_table.cc 95 2025-02-10 Mon 02:02:49:671.921 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459649259264, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating virtual-network-network-ipam controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:671.934 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459649259264, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating virtual-network-network-ipam controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:671.948 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459649259264, Pid 109271]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating service-appliance-set:default-global-system-config:opencontrail controller/src/ifmap/ifmap_server_table.cc 95 2025-02-10 Mon 02:02:49:671.963 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459649259264, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-service-appliance-set controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:671.976 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459649259264, Pid 109271]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating role-config:default-global-system-config:juniper-srx:basic controller/src/ifmap/ifmap_server_table.cc 95 2025-02-10 Mon 02:02:49:671.989 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459649259264, Pid 109271]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating node-profile:default-global-system-config:juniper-srx controller/src/ifmap/ifmap_server_table.cc 95 2025-02-10 Mon 02:02:49:672.000 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459649259264, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating node-profile-role-config controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:672.012 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459649259264, Pid 109271]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating job-template:default-global-system-config:fabric_config_template controller/src/ifmap/ifmap_server_table.cc 95 2025-02-10 Mon 02:02:49:672.036 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459649259264, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-job-template controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:672.050 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459649259264, Pid 109271]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating device-functional-group:default-domain:default-project:L3-Server-Leaf-with-Optimized-Multicast controller/src/ifmap/ifmap_server_table.cc 95 2025-02-10 Mon 02:02:49:672.067 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459649259264, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating project-device-functional-group controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:672.083 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459649259264, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating device-functional-group-physical-role controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:672.095 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459649259264, Pid 109271]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating overlay-role:default-global-system-config:crb-access controller/src/ifmap/ifmap_server_table.cc 95 2025-02-10 Mon 02:02:49:673.779 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459649259264, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-overlay-role controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:673.817 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459649259264, Pid 109271]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating device-functional-group:default-domain:default-project:L3-Gateway-Spine controller/src/ifmap/ifmap_server_table.cc 95 2025-02-10 Mon 02:02:49:673.849 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459649259264, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating project-device-functional-group controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:673.881 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459649259264, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating device-functional-group-physical-role controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:673.896 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459649259264, Pid 109271]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating role-definition:default-global-system-config:crb-gateway-leaf controller/src/ifmap/ifmap_server_table.cc 95 2025-02-10 Mon 02:02:49:673.914 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459649259264, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-role-definition controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:673.934 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459649259264, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:673.952 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459649259264, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:673.968 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459649259264, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:673.982 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459649259264, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:673.997 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459649259264, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:674.011 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459649259264, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:674.026 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459649259264, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:674.039 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459649259264, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:674.052 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459649259264, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:674.066 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459649259264, Pid 109271]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating overlay-role:default-global-system-config:crb-gateway controller/src/ifmap/ifmap_server_table.cc 95 2025-02-10 Mon 02:02:49:674.083 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459649259264, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-overlay-role controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:674.098 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459649259264, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-physical-role controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:674.112 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459649259264, Pid 109271]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating alarm:default-global-system-config:system-defined-process-connectivity controller/src/ifmap/ifmap_server_table.cc 95 2025-02-10 Mon 02:02:49:674.130 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459649259264, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-alarm controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:674.145 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459649259264, Pid 109271]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating analytics-snmp-node:default-global-system-config:cn-jenkins-deploy-platform-ansible-os-2705-1. controller/src/ifmap/ifmap_server_table.cc 95 2025-02-10 Mon 02:02:49:674.162 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459649259264, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-analytics-snmp-node controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:674.174 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459649259264, Pid 109271]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating node-profile:default-global-system-config:juniper-qfx10k-lean controller/src/ifmap/ifmap_server_table.cc 95 2025-02-10 Mon 02:02:49:674.193 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459649259264, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-node-profile controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:674.216 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459649259264, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating node-profile-job-template controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:674.229 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459649259264, Pid 109271]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating alarm:default-global-system-config:system-defined-xmpp-connectivity controller/src/ifmap/ifmap_server_table.cc 95 2025-02-10 Mon 02:02:49:674.246 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459649259264, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-alarm controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:674.462 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-02-10 Mon 02:02:49:674.892 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_WARN]: ConfigurationMalformedReferenceWarningLog: bgp_peering No entry in metadata map bgp_router 2e73e8bd-4700-478d-a9c5-e774f00a259b controller/src/ifmap/client/config_json_parser.cc 223 2025-02-10 Mon 02:02:49:674.906 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_WARN]: ConfigurationMalformedReferenceWarningLog: bgp_peering No entry in metadata map bgp_router 2e73e8bd-4700-478d-a9c5-e774f00a259b controller/src/ifmap/client/config_json_parser.cc 223 2025-02-10 Mon 02:02:49:675.956 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_WARN]: ConfigurationMalformedReferenceWarningLog: instance_target No entry in metadata map routing_instance ef9c1bc1-5e42-46ee-a520-d4ea98936273 controller/src/ifmap/client/config_json_parser.cc 223 2025-02-10 Mon 02:02:49:676.117 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating hardware:juniper-srx5800 controller/src/ifmap/ifmap_server_table.cc 95 2025-02-10 Mon 02:02:49:676.133 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating job-template:default-global-system-config:maintenance_mode_activate_template controller/src/ifmap/ifmap_server_table.cc 95 2025-02-10 Mon 02:02:49:676.158 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-job-template controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:676.176 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating network-ipam:default-domain:default-project:service-chain-flat-ipam controller/src/ifmap/ifmap_server_table.cc 95 2025-02-10 Mon 02:02:49:676.201 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating project-network-ipam controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:676.220 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating tag:label=fabric-as-number controller/src/ifmap/ifmap_server_table.cc 95 2025-02-10 Mon 02:02:49:676.242 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating tag-type:label controller/src/ifmap/ifmap_server_table.cc 95 2025-02-10 Mon 02:02:49:676.255 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating tag-tag-type controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:676.268 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating hardware:juniper-mx80 controller/src/ifmap/ifmap_server_table.cc 95 2025-02-10 Mon 02:02:49:676.280 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating alarm:default-global-system-config:system-defined-conf-incorrect controller/src/ifmap/ifmap_server_table.cc 95 2025-02-10 Mon 02:02:49:676.300 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-alarm controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:676.313 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating access-control-list:default-domain:default-project:__no_rule__:ingress-access-control-list controller/src/ifmap/ifmap_server_table.cc 95 2025-02-10 Mon 02:02:49:676.333 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating security-group-access-control-list controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:676.345 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating role-config:default-global-system-config:juniper-mx:basic controller/src/ifmap/ifmap_server_table.cc 95 2025-02-10 Mon 02:02:49:676.358 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating node-profile:default-global-system-config:juniper-mx controller/src/ifmap/ifmap_server_table.cc 95 2025-02-10 Mon 02:02:49:676.370 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating node-profile-role-config controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:676.387 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating job-template:default-global-system-config:device_import_template controller/src/ifmap/ifmap_server_table.cc 95 2025-02-10 Mon 02:02:49:676.407 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-job-template controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:676.422 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating config-node:default-global-system-config:cn-jenkins-deploy-platform-ansible-os-2705-3. controller/src/ifmap/ifmap_server_table.cc 95 2025-02-10 Mon 02:02:49:676.438 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-config-node controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:676.451 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating analytics-alarm-node:default-global-system-config:cn-jenkins-deploy-platform-ansible-os-2705-1. controller/src/ifmap/ifmap_server_table.cc 95 2025-02-10 Mon 02:02:49:676.466 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-analytics-alarm-node controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:676.478 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating role-config:default-global-system-config:juniper-qfx10k:basic controller/src/ifmap/ifmap_server_table.cc 95 2025-02-10 Mon 02:02:49:676.492 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating node-profile:default-global-system-config:juniper-qfx10k controller/src/ifmap/ifmap_server_table.cc 95 2025-02-10 Mon 02:02:49:676.503 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating node-profile-role-config controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:676.515 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating role-definition:default-global-system-config:dci-gateway-leaf controller/src/ifmap/ifmap_server_table.cc 95 2025-02-10 Mon 02:02:49:676.532 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-role-definition controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:676.549 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:676.565 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:676.579 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:676.593 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:676.607 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:676.626 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:676.640 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:676.654 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:676.669 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating overlay-role:default-global-system-config:dci-gateway controller/src/ifmap/ifmap_server_table.cc 95 2025-02-10 Mon 02:02:49:676.680 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-overlay-role controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:676.695 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-physical-role controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:676.707 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating overlay-role:default-global-system-config:route-reflector controller/src/ifmap/ifmap_server_table.cc 95 2025-02-10 Mon 02:02:49:676.722 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-overlay-role controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:676.734 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating alarm:default-global-system-config:system-defined-partial-sysinfo controller/src/ifmap/ifmap_server_table.cc 95 2025-02-10 Mon 02:02:49:676.750 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-alarm controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:676.765 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating overlay-role:default-global-system-config:ar-replicator controller/src/ifmap/ifmap_server_table.cc 95 2025-02-10 Mon 02:02:49:676.779 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-overlay-role controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:676.791 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating hardware:juniper-mx960 controller/src/ifmap/ifmap_server_table.cc 95 2025-02-10 Mon 02:02:49:676.802 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating job-template:default-global-system-config:search_ip_mac_template controller/src/ifmap/ifmap_server_table.cc 95 2025-02-10 Mon 02:02:49:676.822 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-job-template controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:676.834 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating policy-management:default-policy-management controller/src/ifmap/ifmap_server_table.cc 95 2025-02-10 Mon 02:02:49:676.855 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:676.878 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating access-control-list:default-domain:default-project:ip-fabric:ip-fabric controller/src/ifmap/ifmap_server_table.cc 95 2025-02-10 Mon 02:02:49:676.933 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating virtual-network:default-domain:default-project:ip-fabric controller/src/ifmap/ifmap_server_table.cc 95 2025-02-10 Mon 02:02:49:676.944 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating virtual-network-access-control-list controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:676.956 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating alarm:default-global-system-config:system-defined-address-mismatch-compute controller/src/ifmap/ifmap_server_table.cc 95 2025-02-10 Mon 02:02:49:676.972 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-alarm controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:676.984 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating hardware:juniper-qfx5120-32c controller/src/ifmap/ifmap_server_table.cc 95 2025-02-10 Mon 02:02:49:676.995 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating hardware:juniper-vmx controller/src/ifmap/ifmap_server_table.cc 95 2025-02-10 Mon 02:02:49:677.005 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating hardware:juniper-qfx5200-32c-32q controller/src/ifmap/ifmap_server_table.cc 95 2025-02-10 Mon 02:02:49:677.019 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:677.033 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating alarm:default-global-system-config:system-defined-vrouter-limit-exceeded controller/src/ifmap/ifmap_server_table.cc 95 2025-02-10 Mon 02:02:49:677.049 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-alarm controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:677.063 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating job-template:default-global-system-config:fabric_onboard_template controller/src/ifmap/ifmap_server_table.cc 95 2025-02-10 Mon 02:02:49:677.083 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-job-template controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:677.095 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating role-definition:default-global-system-config:rr-leaf controller/src/ifmap/ifmap_server_table.cc 95 2025-02-10 Mon 02:02:49:677.108 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-role-definition controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:677.127 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:677.142 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:677.981 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459867805440, Pid 109271]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-02-10 Mon 02:02:49:678.175 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459867805440, Pid 109271]: IFMap [SYS_WARN]: ConfigurationMalformedReferenceWarningLog: instance_target No entry in metadata map routing_instance 05a57f6e-2903-4a97-82c3-373483b53d39 controller/src/ifmap/client/config_json_parser.cc 223 2025-02-10 Mon 02:02:49:678.500 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459867805440, Pid 109271]: IFMap [SYS_WARN]: ConfigurationMalformedReferenceWarningLog: overlay_role Link name is empty node_profile 414dda90-89e8-43e2-8475-cfa8f88b5213 controller/src/ifmap/client/config_json_parser.cc 217 2025-02-10 Mon 02:02:49:678.512 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459867805440, Pid 109271]: IFMap [SYS_WARN]: ConfigurationMalformedReferenceWarningLog: physical_role Link name is empty node_profile 414dda90-89e8-43e2-8475-cfa8f88b5213 controller/src/ifmap/client/config_json_parser.cc 217 2025-02-10 Mon 02:02:49:679.275 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459867805440, Pid 109271]: IFMap [SYS_WARN]: ConfigurationMalformedReferenceWarningLog: bgp_peering No entry in metadata map bgp_router c132f712-0f40-4d84-8526-ff746c5eb961 controller/src/ifmap/client/config_json_parser.cc 223 2025-02-10 Mon 02:02:49:679.288 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459867805440, Pid 109271]: IFMap [SYS_WARN]: ConfigurationMalformedReferenceWarningLog: bgp_peering No entry in metadata map bgp_router c132f712-0f40-4d84-8526-ff746c5eb961 controller/src/ifmap/client/config_json_parser.cc 223 2025-02-10 Mon 02:02:49:682.391 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459636664064, Pid 109271]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-02-10 Mon 02:02:49:683.397 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459636664064, Pid 109271]: IFMap [SYS_WARN]: ConfigurationMalformedReferenceWarningLog: instance_target No entry in metadata map routing_instance 8e048815-bd6e-445b-988f-02657b027075 controller/src/ifmap/client/config_json_parser.cc 223 2025-02-10 Mon 02:02:49:683.653 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459636664064, Pid 109271]: IFMap [SYS_WARN]: ConfigurationMalformedReferenceWarningLog: overlay_role Link name is empty node_profile c332b70e-0113-4787-a186-5e1f2afcbdb1 controller/src/ifmap/client/config_json_parser.cc 217 2025-02-10 Mon 02:02:49:684.340 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459872003840, Pid 109271]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-02-10 Mon 02:02:49:685.430 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459872003840, Pid 109271]: IFMap [SYS_WARN]: ConfigurationMalformedReferenceWarningLog: instance_target No entry in metadata map routing_instance 9a498f7c-34e7-4f05-b457-f23549c7aac4 controller/src/ifmap/client/config_json_parser.cc 223 2025-02-10 Mon 02:02:49:686.295 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459640862464, Pid 109271]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-02-10 Mon 02:02:49:687.995 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459645060864, Pid 109271]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-02-10 Mon 02:02:49:688.221 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459645060864, Pid 109271]: IFMap [SYS_WARN]: ConfigurationMalformedReferenceWarningLog: instance_target No entry in metadata map routing_instance 08b0e674-2c2c-40a8-89d9-e8af64d04f6e controller/src/ifmap/client/config_json_parser.cc 223 2025-02-10 Mon 02:02:49:689.926 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:689.961 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:689.982 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-overlay-role controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:690.000 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-physical-role controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:690.017 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating telemetry-profile:default-domain:default-project:default-telemetry-profile-2 controller/src/ifmap/ifmap_server_table.cc 95 2025-02-10 Mon 02:02:49:690.037 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating project-telemetry-profile controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:690.055 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating sflow-profile:default-domain:default-project:sflow-fabric-interfaces controller/src/ifmap/ifmap_server_table.cc 95 2025-02-10 Mon 02:02:49:690.069 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating telemetry-profile-sflow-profile controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:690.084 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating role-definition:default-global-system-config:pnf-servicechain-pnf controller/src/ifmap/ifmap_server_table.cc 95 2025-02-10 Mon 02:02:49:690.113 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-role-definition controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:690.132 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:690.150 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-overlay-role controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:690.166 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating physical-role:default-global-system-config:pnf controller/src/ifmap/ifmap_server_table.cc 95 2025-02-10 Mon 02:02:49:690.188 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-physical-role controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:690.203 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating hardware:juniper-qfx5220-128c controller/src/ifmap/ifmap_server_table.cc 95 2025-02-10 Mon 02:02:49:690.215 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating fabric:default-global-system-config:default-fabric controller/src/ifmap/ifmap_server_table.cc 95 2025-02-10 Mon 02:02:49:690.234 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-fabric controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:690.512 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:690.534 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:690.549 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating role-definition:default-global-system-config:pnf-servicechain-spine controller/src/ifmap/ifmap_server_table.cc 95 2025-02-10 Mon 02:02:49:690.567 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-role-definition controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:690.584 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:690.601 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-overlay-role controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:690.617 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-physical-role controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:690.632 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating virtual-network:default-domain:default-project:dci-network controller/src/ifmap/ifmap_server_table.cc 95 2025-02-10 Mon 02:02:49:690.672 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating project-virtual-network controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:690.690 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating virtual-router:default-global-system-config:an-jenkins-deploy-platform-ansible-os-2705-1 controller/src/ifmap/ifmap_server_table.cc 95 2025-02-10 Mon 02:02:49:690.711 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-virtual-router controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:690.725 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating tag:label=fabric-management-ip controller/src/ifmap/ifmap_server_table.cc 95 2025-02-10 Mon 02:02:49:690.749 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating tag-tag-type controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:690.762 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating global-vrouter-config:default-global-system-config:default-global-vrouter-config controller/src/ifmap/ifmap_server_table.cc 95 2025-02-10 Mon 02:02:49:690.783 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-global-vrouter-config controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:690.802 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:690.817 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating job-template:default-global-system-config:ztp_template controller/src/ifmap/ifmap_server_table.cc 95 2025-02-10 Mon 02:02:49:690.840 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-job-template controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:690.860 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating domain-project controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:690.883 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating node-profile:default-global-system-config:juniper-qfx5k controller/src/ifmap/ifmap_server_table.cc 95 2025-02-10 Mon 02:02:49:690.901 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-node-profile controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:690.916 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating hardware:juniper-qfx5110-32q controller/src/ifmap/ifmap_server_table.cc 95 2025-02-10 Mon 02:02:49:690.927 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating node-profile-hardware controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:690.940 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating hardware:juniper-qfx5110-48s-4c controller/src/ifmap/ifmap_server_table.cc 95 2025-02-10 Mon 02:02:49:690.951 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating node-profile-hardware controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:690.966 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating node-profile-job-template controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:690.979 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating hardware:juniper-mx2008 controller/src/ifmap/ifmap_server_table.cc 95 2025-02-10 Mon 02:02:49:690.991 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating job-template:default-global-system-config:rma_activate_template controller/src/ifmap/ifmap_server_table.cc 95 2025-02-10 Mon 02:02:49:691.017 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-job-template controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:691.031 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating hardware:juniper-qfx5100-48t-6q controller/src/ifmap/ifmap_server_table.cc 95 2025-02-10 Mon 02:02:49:691.042 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating hardware:juniper-qfx5100-48sh-afo controller/src/ifmap/ifmap_server_table.cc 95 2025-02-10 Mon 02:02:49:691.052 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating sflow-profile:default-domain:default-project:sflow-access-interfaces controller/src/ifmap/ifmap_server_table.cc 95 2025-02-10 Mon 02:02:49:691.074 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating project-sflow-profile controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:691.088 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating role-definition:default-global-system-config:dc-gateway-leaf controller/src/ifmap/ifmap_server_table.cc 95 2025-02-10 Mon 02:02:49:691.105 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-role-definition controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:691.121 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:691.137 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:691.152 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:691.167 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:691.181 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:691.195 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:691.210 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:691.224 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:691.240 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating overlay-role:default-global-system-config:dc-gateway controller/src/ifmap/ifmap_server_table.cc 95 2025-02-10 Mon 02:02:49:691.250 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-overlay-role controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:691.269 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-physical-role controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:691.282 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating tag-type:neutron_tag controller/src/ifmap/ifmap_server_table.cc 95 2025-02-10 Mon 02:02:49:691.295 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating alarm:default-global-system-config:system-defined-vrouter-interface controller/src/ifmap/ifmap_server_table.cc 95 2025-02-10 Mon 02:02:49:691.316 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-alarm controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:691.330 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating hardware:juniper-jnp10008 controller/src/ifmap/ifmap_server_table.cc 95 2025-02-10 Mon 02:02:49:691.342 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating analytics-node:default-global-system-config:cn-jenkins-deploy-platform-ansible-os-2705-3. controller/src/ifmap/ifmap_server_table.cc 95 2025-02-10 Mon 02:02:49:691.360 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-analytics-node controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:691.380 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-overlay-role controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:691.393 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating job-template:default-global-system-config:show_config_template controller/src/ifmap/ifmap_server_table.cc 95 2025-02-10 Mon 02:02:49:691.414 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-job-template controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:691.428 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating job-template:default-global-system-config:hardware_inventory_template controller/src/ifmap/ifmap_server_table.cc 95 2025-02-10 Mon 02:02:49:691.449 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-job-template controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:691.461 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating hardware:juniper-srx240h-poe controller/src/ifmap/ifmap_server_table.cc 95 2025-02-10 Mon 02:02:49:691.474 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating feature-config:default-global-system-config:erb-leaf:l3-gateway controller/src/ifmap/ifmap_server_table.cc 95 2025-02-10 Mon 02:02:49:691.487 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating role-definition:default-global-system-config:erb-leaf controller/src/ifmap/ifmap_server_table.cc 95 2025-02-10 Mon 02:02:49:691.498 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature-config controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:691.521 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:691.534 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating alarm:default-global-system-config:system-defined-vrouter-l3mh-physical-interface-link-down controller/src/ifmap/ifmap_server_table.cc 95 2025-02-10 Mon 02:02:49:691.552 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-alarm controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:691.567 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating role-config:default-global-system-config:juniper-qfx5220:basic controller/src/ifmap/ifmap_server_table.cc 95 2025-02-10 Mon 02:02:49:691.580 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating node-profile:default-global-system-config:juniper-qfx5220 controller/src/ifmap/ifmap_server_table.cc 95 2025-02-10 Mon 02:02:49:691.591 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating node-profile-role-config controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:691.604 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating job-template:default-global-system-config:test_overlay_connectivity_template controller/src/ifmap/ifmap_server_table.cc 95 2025-02-10 Mon 02:02:49:691.625 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-job-template controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:691.637 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating hardware:juniper-qfx10002-72q controller/src/ifmap/ifmap_server_table.cc 95 2025-02-10 Mon 02:02:49:691.656 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-physical-role controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:691.669 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating role-definition:default-global-system-config:ar-replicator-leaf controller/src/ifmap/ifmap_server_table.cc 95 2025-02-10 Mon 02:02:49:691.685 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-role-definition controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:691.702 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:691.716 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating feature:default-global-system-config:assisted-replicator controller/src/ifmap/ifmap_server_table.cc 95 2025-02-10 Mon 02:02:49:691.727 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:691.745 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:691.761 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:691.776 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-overlay-role controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:691.791 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-physical-role controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:691.806 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating hardware:juniper-qfx5100-48th-afo controller/src/ifmap/ifmap_server_table.cc 95 2025-02-10 Mon 02:02:49:691.817 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating hardware:juniper-qfx5120-48t-6c controller/src/ifmap/ifmap_server_table.cc 95 2025-02-10 Mon 02:02:49:691.827 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating telemetry-profile:default-domain:default-project:default-telemetry-profile-3 controller/src/ifmap/ifmap_server_table.cc 95 2025-02-10 Mon 02:02:49:691.843 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating project-telemetry-profile controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:691.860 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating telemetry-profile-sflow-profile controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:691.887 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:691.901 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating hardware:juniper-mx80-48t controller/src/ifmap/ifmap_server_table.cc 95 2025-02-10 Mon 02:02:49:691.913 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating node-profile:default-global-system-config:juniper-qfx5120 controller/src/ifmap/ifmap_server_table.cc 95 2025-02-10 Mon 02:02:49:691.931 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-node-profile controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:691.949 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating node-profile-hardware controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:691.964 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating node-profile-hardware controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:691.979 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating node-profile-hardware controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:691.995 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating node-profile-job-template controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:692.011 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating hardware:juniper-jnp10016 controller/src/ifmap/ifmap_server_table.cc 95 2025-02-10 Mon 02:02:49:692.023 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating device-functional-group:default-domain:default-project:Centrally-Routed-Border-Spine-With-Optimized-Multicast controller/src/ifmap/ifmap_server_table.cc 95 2025-02-10 Mon 02:02:49:692.041 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating project-device-functional-group controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:692.059 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating device-functional-group-physical-role controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:692.072 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating config-database-node:default-global-system-config:cn-jenkins-deploy-platform-ansible-os-2705-2. controller/src/ifmap/ifmap_server_table.cc 95 2025-02-10 Mon 02:02:49:692.088 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-config-database-node controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:692.101 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating hardware:juniper-srx4200 controller/src/ifmap/ifmap_server_table.cc 95 2025-02-10 Mon 02:02:49:692.113 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating analytics-snmp-node:default-global-system-config:cn-jenkins-deploy-platform-ansible-os-2705-3. controller/src/ifmap/ifmap_server_table.cc 95 2025-02-10 Mon 02:02:49:692.130 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-analytics-snmp-node controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:692.151 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating project-network-ipam controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:692.165 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating virtual-machine-interface:default-global-system-config:an-jenkins-deploy-platform-ansible-os-2705-2:vhost0 controller/src/ifmap/ifmap_server_table.cc 95 2025-02-10 Mon 02:02:49:692.184 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating virtual-router-virtual-machine-interface controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:692.205 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating virtual-machine-interface-virtual-network controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:692.217 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating overlay-role:default-global-system-config:crb-mcast-gateway controller/src/ifmap/ifmap_server_table.cc 95 2025-02-10 Mon 02:02:49:692.232 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-overlay-role controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:692.248 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating job-template:default-global-system-config:vcenter_import_template controller/src/ifmap/ifmap_server_table.cc 95 2025-02-10 Mon 02:02:49:692.269 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-job-template controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:692.285 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating hardware:juniper-qfx5100e-48s-6q controller/src/ifmap/ifmap_server_table.cc 95 2025-02-10 Mon 02:02:49:692.296 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating config-node:default-global-system-config:cn-jenkins-deploy-platform-ansible-os-2705-1. controller/src/ifmap/ifmap_server_table.cc 95 2025-02-10 Mon 02:02:49:692.311 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-config-node controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:692.324 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating hardware:juniper-mx2020 controller/src/ifmap/ifmap_server_table.cc 95 2025-02-10 Mon 02:02:49:692.336 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating application-policy-set:default-domain:default-project:default-application-policy-set controller/src/ifmap/ifmap_server_table.cc 95 2025-02-10 Mon 02:02:49:692.352 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating project-application-policy-set controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:692.364 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating feature-config:default-global-system-config:ar-client-spine:assisted-replicator controller/src/ifmap/ifmap_server_table.cc 95 2025-02-10 Mon 02:02:49:692.378 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating role-definition:default-global-system-config:ar-client-spine controller/src/ifmap/ifmap_server_table.cc 95 2025-02-10 Mon 02:02:49:692.391 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature-config controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:692.404 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating hardware:juniper-mx204 controller/src/ifmap/ifmap_server_table.cc 95 2025-02-10 Mon 02:02:49:692.415 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating hardware:juniper-qfx10016 controller/src/ifmap/ifmap_server_table.cc 95 2025-02-10 Mon 02:02:49:692.425 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating hardware:juniper-qfx10002-60c controller/src/ifmap/ifmap_server_table.cc 95 2025-02-10 Mon 02:02:49:692.437 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating analytics-node:default-global-system-config:cn-jenkins-deploy-platform-ansible-os-2705-2. controller/src/ifmap/ifmap_server_table.cc 95 2025-02-10 Mon 02:02:49:692.453 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-analytics-node controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:692.469 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating tag:label=fabric-ebgp-as-number controller/src/ifmap/ifmap_server_table.cc 95 2025-02-10 Mon 02:02:49:692.487 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating tag-tag-type controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:692.502 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating service-appliance-set:default-global-system-config:native controller/src/ifmap/ifmap_server_table.cc 95 2025-02-10 Mon 02:02:49:692.518 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-service-appliance-set controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:692.531 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating role-definition:default-global-system-config:crb-mcast-gateway-leaf controller/src/ifmap/ifmap_server_table.cc 95 2025-02-10 Mon 02:02:49:692.546 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-role-definition controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:692.562 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:692.581 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:692.598 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:692.614 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:692.629 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:692.645 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:692.659 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:692.672 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:692.687 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-overlay-role controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:692.704 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-physical-role controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:692.721 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-overlay-role controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:692.740 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating device-functional-group:default-domain:default-project:Border-Spine-in-Edge-Routed controller/src/ifmap/ifmap_server_table.cc 95 2025-02-10 Mon 02:02:49:692.758 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating project-device-functional-group controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:692.774 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating device-functional-group-physical-role controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:692.787 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating role-definition:default-global-system-config:ar-replicator-spine controller/src/ifmap/ifmap_server_table.cc 95 2025-02-10 Mon 02:02:49:692.801 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-role-definition controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:692.819 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:692.833 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:692.848 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:692.862 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:692.887 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-overlay-role controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:692.902 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-physical-role controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:692.915 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating alarm:default-global-system-config:system-defined-core-files controller/src/ifmap/ifmap_server_table.cc 95 2025-02-10 Mon 02:02:49:692.934 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-alarm controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:692.947 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating hardware:juniper-mx480 controller/src/ifmap/ifmap_server_table.cc 95 2025-02-10 Mon 02:02:49:692.959 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating job-template:default-global-system-config:maintenance_mode_deactivate_template controller/src/ifmap/ifmap_server_table.cc 95 2025-02-10 Mon 02:02:49:692.980 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-job-template controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:693.006 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-node-profile controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:693.021 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating hardware:juniper-vqfx-10000 controller/src/ifmap/ifmap_server_table.cc 95 2025-02-10 Mon 02:02:49:693.032 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating node-profile-hardware controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:693.047 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating node-profile-hardware controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:693.064 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating node-profile-hardware controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:693.078 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating node-profile-hardware controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:693.091 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating hardware:juniper-qfx10008 controller/src/ifmap/ifmap_server_table.cc 95 2025-02-10 Mon 02:02:49:693.102 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating node-profile-hardware controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:693.119 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating hardware:juniper-qfx10002-36q controller/src/ifmap/ifmap_server_table.cc 95 2025-02-10 Mon 02:02:49:693.129 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating node-profile-hardware controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:693.144 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating node-profile-job-template controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:693.541 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-overlay-role controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:693.561 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating job-template:default-global-system-config:discover_device_template controller/src/ifmap/ifmap_server_table.cc 95 2025-02-10 Mon 02:02:49:693.581 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-job-template controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:693.598 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating hardware:juniper-srx4100 controller/src/ifmap/ifmap_server_table.cc 95 2025-02-10 Mon 02:02:49:693.612 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating alarm:default-global-system-config:system-defined-prouter-tsn-connectivity controller/src/ifmap/ifmap_server_table.cc 95 2025-02-10 Mon 02:02:49:693.633 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-alarm controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:693.653 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-node-profile controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:693.670 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating hardware:juniper-mx240 controller/src/ifmap/ifmap_server_table.cc 95 2025-02-10 Mon 02:02:49:693.680 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating node-profile-hardware controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:693.695 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating node-profile-hardware controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:693.708 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating hardware:juniper-mx2010 controller/src/ifmap/ifmap_server_table.cc 95 2025-02-10 Mon 02:02:49:693.719 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating node-profile-hardware controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:693.734 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating node-profile-hardware controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:693.749 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating node-profile-hardware controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:693.763 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating node-profile-hardware controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:693.778 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating node-profile-hardware controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:693.793 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating node-profile-hardware controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:693.807 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating node-profile-hardware controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:693.821 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating node-profile-hardware controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:693.838 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating hardware:juniper-mx10003 controller/src/ifmap/ifmap_server_table.cc 95 2025-02-10 Mon 02:02:49:693.849 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating node-profile-hardware controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:693.863 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating node-profile-hardware controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:693.889 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating node-profile-hardware controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:693.903 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating node-profile-job-template controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:693.924 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating project-sflow-profile controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:693.939 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating database-node:default-global-system-config:cn-jenkins-deploy-platform-ansible-os-2705-3. controller/src/ifmap/ifmap_server_table.cc 95 2025-02-10 Mon 02:02:49:693.955 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-database-node controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:693.979 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating job-template:default-global-system-config:show_mac_mob_template controller/src/ifmap/ifmap_server_table.cc 95 2025-02-10 Mon 02:02:49:694.001 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-job-template controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:694.015 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating overlay-role:default-global-system-config:ar-client controller/src/ifmap/ifmap_server_table.cc 95 2025-02-10 Mon 02:02:49:694.029 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-overlay-role controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:694.045 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating grpc-profile:default-domain:default-project:grpc-default-profile controller/src/ifmap/ifmap_server_table.cc 95 2025-02-10 Mon 02:02:49:694.062 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating project-grpc-profile controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:694.080 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating tag:label=fabric-loopback-ip controller/src/ifmap/ifmap_server_table.cc 95 2025-02-10 Mon 02:02:49:694.099 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating tag-tag-type controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:694.115 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating role-definition:default-global-system-config:dci-gateway-spine controller/src/ifmap/ifmap_server_table.cc 95 2025-02-10 Mon 02:02:49:694.131 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-role-definition controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:694.148 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:694.164 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:694.186 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:694.201 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:694.216 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:694.233 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:694.248 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:694.262 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:694.277 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-overlay-role controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:694.296 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-physical-role controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:694.316 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating project-security-group controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:694.335 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:694.362 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-node-profile controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:694.378 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating node-profile-hardware controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:694.393 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating node-profile-hardware controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:694.410 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating node-profile-hardware controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:694.425 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating node-profile-hardware controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:694.438 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating node-profile-hardware controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:694.452 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating node-profile-hardware controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:694.472 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating node-profile-hardware controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:694.486 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating node-profile-hardware controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:694.499 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating hardware:juniper-srx4600 controller/src/ifmap/ifmap_server_table.cc 95 2025-02-10 Mon 02:02:49:694.509 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating node-profile-hardware controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:694.526 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating node-profile-job-template controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:694.539 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating device-functional-group:default-domain:default-project:Lean-Spine-with-Route-Reflector controller/src/ifmap/ifmap_server_table.cc 95 2025-02-10 Mon 02:02:49:694.556 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating project-device-functional-group controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:694.573 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating device-functional-group-physical-role controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:694.588 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating job-template:default-global-system-config:show_chassis_info_template controller/src/ifmap/ifmap_server_table.cc 95 2025-02-10 Mon 02:02:49:694.609 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-job-template controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:694.623 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating hardware:juniper-qfx5100-48sh-afi controller/src/ifmap/ifmap_server_table.cc 95 2025-02-10 Mon 02:02:49:694.639 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating network-ipam:default-domain:default-project:default-network-ipam controller/src/ifmap/ifmap_server_table.cc 95 2025-02-10 Mon 02:02:49:694.653 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating project-network-ipam controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:694.666 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating role-definition:default-global-system-config:lean-spine controller/src/ifmap/ifmap_server_table.cc 95 2025-02-10 Mon 02:02:49:694.684 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-role-definition controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:694.701 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:694.720 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:694.735 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:694.751 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating overlay-role:default-global-system-config:lean controller/src/ifmap/ifmap_server_table.cc 95 2025-02-10 Mon 02:02:49:694.762 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-overlay-role controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:694.778 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-physical-role controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:694.791 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating job-template:default-global-system-config:discover_os_computes_template controller/src/ifmap/ifmap_server_table.cc 95 2025-02-10 Mon 02:02:49:694.813 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-job-template controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:694.827 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating job-template:default-global-system-config:role_assignment_template controller/src/ifmap/ifmap_server_table.cc 95 2025-02-10 Mon 02:02:49:694.845 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-job-template controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:694.858 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating job-template:default-global-system-config:cli_sync_template controller/src/ifmap/ifmap_server_table.cc 95 2025-02-10 Mon 02:02:49:694.889 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-job-template controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:694.902 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating role-definition:default-global-system-config:crb-gateway-spine controller/src/ifmap/ifmap_server_table.cc 95 2025-02-10 Mon 02:02:49:694.917 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-role-definition controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:694.936 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:694.952 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:694.967 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:694.985 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:695.002 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:695.017 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:695.031 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:695.045 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:695.062 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:695.078 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-overlay-role controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:695.092 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-physical-role controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:695.108 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating alarm:default-global-system-config:system-defined-prouter-connectivity controller/src/ifmap/ifmap_server_table.cc 95 2025-02-10 Mon 02:02:49:695.127 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-alarm controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:695.140 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating role-definition:default-global-system-config:crb-access-spine controller/src/ifmap/ifmap_server_table.cc 95 2025-02-10 Mon 02:02:49:695.154 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-role-definition controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:695.173 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:695.188 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:695.202 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:695.217 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:695.234 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:695.252 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:695.266 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:695.281 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-overlay-role controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:695.299 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-physical-role controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:695.311 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating global-qos-config:default-global-system-config:default-global-qos-config controller/src/ifmap/ifmap_server_table.cc 95 2025-02-10 Mon 02:02:49:695.325 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-global-qos-config controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:695.342 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating service-template:default-domain:nat-template controller/src/ifmap/ifmap_server_table.cc 95 2025-02-10 Mon 02:02:49:695.360 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating domain-service-template controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:695.373 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating hardware:juniper-qfx5100e-48t-6q controller/src/ifmap/ifmap_server_table.cc 95 2025-02-10 Mon 02:02:49:695.387 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating database-node:default-global-system-config:cn-jenkins-deploy-platform-ansible-os-2705-1. controller/src/ifmap/ifmap_server_table.cc 95 2025-02-10 Mon 02:02:49:695.402 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-database-node controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:695.416 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating snmp-profile:default-domain:default-project:snmp-default-profile controller/src/ifmap/ifmap_server_table.cc 95 2025-02-10 Mon 02:02:49:695.432 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating project-snmp-profile controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:695.456 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-node-profile controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:695.472 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating node-profile-hardware controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:695.487 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating node-profile-hardware controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:695.506 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating node-profile-job-template controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:695.522 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating job-template:default-global-system-config:container_cleanup_template controller/src/ifmap/ifmap_server_table.cc 95 2025-02-10 Mon 02:02:49:695.542 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-job-template controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:695.555 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating telemetry-profile:default-domain:default-project:default-telemetry-profile-5 controller/src/ifmap/ifmap_server_table.cc 95 2025-02-10 Mon 02:02:49:695.573 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating project-telemetry-profile controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:695.589 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating telemetry-profile-snmp-profile controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:695.601 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating alarm:default-global-system-config:system-defined-disk-usage-high controller/src/ifmap/ifmap_server_table.cc 95 2025-02-10 Mon 02:02:49:695.620 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-alarm controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:695.633 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating hardware:juniper-qfx5100-48s-6q controller/src/ifmap/ifmap_server_table.cc 95 2025-02-10 Mon 02:02:49:695.647 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating alarm:default-global-system-config:system-defined-xmpp-close-reason controller/src/ifmap/ifmap_server_table.cc 95 2025-02-10 Mon 02:02:49:695.663 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-alarm controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:695.678 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating physical-role:default-global-system-config:superspine controller/src/ifmap/ifmap_server_table.cc 95 2025-02-10 Mon 02:02:49:695.693 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-physical-role controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:695.706 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating alarm:default-global-system-config:system-defined-address-mismatch-control controller/src/ifmap/ifmap_server_table.cc 95 2025-02-10 Mon 02:02:49:695.726 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-alarm controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:695.739 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating telemetry-profile:default-domain:default-project:default-telemetry-profile-4 controller/src/ifmap/ifmap_server_table.cc 95 2025-02-10 Mon 02:02:49:695.756 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating project-telemetry-profile controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:695.772 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating telemetry-profile-grpc-profile controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:695.793 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-role-definition controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:695.809 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:695.824 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:695.841 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:695.857 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-overlay-role controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:695.879 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-physical-role controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:695.892 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating tag:label=fabric-overlay-loopback-ip controller/src/ifmap/ifmap_server_table.cc 95 2025-02-10 Mon 02:02:49:695.911 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating tag-tag-type controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:695.926 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating job-template:default-global-system-config:show_interface_details_template controller/src/ifmap/ifmap_server_table.cc 95 2025-02-10 Mon 02:02:49:695.947 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-job-template controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:695.968 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-overlay-role controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:695.982 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating virtual-machine-interface:default-global-system-config:an-jenkins-deploy-platform-ansible-os-2705-1:vhost0 controller/src/ifmap/ifmap_server_table.cc 95 2025-02-10 Mon 02:02:49:696.002 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating virtual-router-virtual-machine-interface controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:696.022 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating virtual-machine-interface-virtual-network controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:696.043 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating job-template:default-global-system-config:hitless_upgrade_strategy_template controller/src/ifmap/ifmap_server_table.cc 95 2025-02-10 Mon 02:02:49:696.064 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-job-template controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:696.080 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating alarm:default-global-system-config:system-defined-conf-elements-inconsistency controller/src/ifmap/ifmap_server_table.cc 95 2025-02-10 Mon 02:02:49:696.098 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-alarm controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:696.111 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating job-template:default-global-system-config:node_profile_template controller/src/ifmap/ifmap_server_table.cc 95 2025-02-10 Mon 02:02:49:696.129 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-job-template controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:696.171 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-node-profile controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:696.189 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating node-profile-hardware controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:696.208 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating node-profile-hardware controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:696.223 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating node-profile-hardware controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:696.238 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating node-profile-hardware controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:696.252 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating node-profile-hardware controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:696.269 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating node-profile-hardware controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:696.284 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating node-profile-hardware controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:696.297 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating hardware:juniper-qfx5100e-24q-2p controller/src/ifmap/ifmap_server_table.cc 95 2025-02-10 Mon 02:02:49:696.307 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating node-profile-hardware controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:696.326 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating hardware:juniper-qfx5100-48th-afi controller/src/ifmap/ifmap_server_table.cc 95 2025-02-10 Mon 02:02:49:696.337 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating node-profile-hardware controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:696.351 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating node-profile-hardware controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:696.365 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating node-profile-hardware controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:696.382 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating node-profile-hardware controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:696.395 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating hardware:juniper-qfx5100-24q-2p controller/src/ifmap/ifmap_server_table.cc 95 2025-02-10 Mon 02:02:49:696.405 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating node-profile-hardware controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:696.419 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating node-profile-hardware controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:696.435 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating node-profile-hardware controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:696.450 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating node-profile-hardware controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:696.463 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating hardware:juniper-qfx5100-48t controller/src/ifmap/ifmap_server_table.cc 95 2025-02-10 Mon 02:02:49:696.473 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating node-profile-hardware controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:696.490 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating node-profile-job-template controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:696.504 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating config-database-node:default-global-system-config:cn-jenkins-deploy-platform-ansible-os-2705-3. controller/src/ifmap/ifmap_server_table.cc 95 2025-02-10 Mon 02:02:49:696.519 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-config-database-node controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:696.537 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating analytics-node:default-global-system-config:cn-jenkins-deploy-platform-ansible-os-2705-1. controller/src/ifmap/ifmap_server_table.cc 95 2025-02-10 Mon 02:02:49:696.553 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-analytics-node controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:696.690 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating alarm:default-global-system-config:system-defined-package-version-mismatch controller/src/ifmap/ifmap_server_table.cc 95 2025-02-10 Mon 02:02:49:696.710 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-alarm controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:696.727 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating tag-type:tier controller/src/ifmap/ifmap_server_table.cc 95 2025-02-10 Mon 02:02:49:696.739 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating alarm:default-global-system-config:system-defined-disk-usage-critical controller/src/ifmap/ifmap_server_table.cc 95 2025-02-10 Mon 02:02:49:696.755 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-alarm controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:696.773 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating config-node:default-global-system-config:cn-jenkins-deploy-platform-ansible-os-2705-2. controller/src/ifmap/ifmap_server_table.cc 95 2025-02-10 Mon 02:02:49:696.788 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-config-node controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:696.807 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-physical-role controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:696.833 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating project-sflow-profile controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:696.854 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-overlay-role controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:696.875 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating tag-type:site controller/src/ifmap/ifmap_server_table.cc 95 2025-02-10 Mon 02:02:49:696.918 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating project-virtual-network controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:696.932 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating analytics-alarm-node:default-global-system-config:cn-jenkins-deploy-platform-ansible-os-2705-3. controller/src/ifmap/ifmap_server_table.cc 95 2025-02-10 Mon 02:02:49:696.950 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-analytics-alarm-node controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:696.965 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating device-functional-group:default-domain:default-project:L3-Storage-Leaf controller/src/ifmap/ifmap_server_table.cc 95 2025-02-10 Mon 02:02:49:696.987 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating project-device-functional-group controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:697.006 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating device-functional-group-physical-role controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:697.022 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating role-config:default-global-system-config:juniper-qfx5k:basic controller/src/ifmap/ifmap_server_table.cc 95 2025-02-10 Mon 02:02:49:697.038 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating node-profile-role-config controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:697.053 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating tag:label=fabric-pnf-servicechain-ip controller/src/ifmap/ifmap_server_table.cc 95 2025-02-10 Mon 02:02:49:697.071 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating tag-tag-type controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:697.087 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating role-definition:default-global-system-config:rr-superspine controller/src/ifmap/ifmap_server_table.cc 95 2025-02-10 Mon 02:02:49:697.102 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-role-definition controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:697.119 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:697.138 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:697.620 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:697.641 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:697.658 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-overlay-role controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:697.679 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-physical-role controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:697.693 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating role-definition:default-global-system-config:crb-access-leaf controller/src/ifmap/ifmap_server_table.cc 95 2025-02-10 Mon 02:02:49:697.710 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-role-definition controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:697.726 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:697.749 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:697.764 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:697.779 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:697.794 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:697.811 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:697.826 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:697.840 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459876202240, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-overlay-role controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:697.866 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459649259264, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-physical-role controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:697.892 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459649259264, Pid 109271]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating tag:label=fabric-peer-ip controller/src/ifmap/ifmap_server_table.cc 95 2025-02-10 Mon 02:02:49:697.911 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459649259264, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating tag-tag-type controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:697.925 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459649259264, Pid 109271]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating role-config:default-global-system-config:juniper-qfx10k-lean:basic controller/src/ifmap/ifmap_server_table.cc 95 2025-02-10 Mon 02:02:49:697.942 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459649259264, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating node-profile-role-config controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:697.955 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459649259264, Pid 109271]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating netconf-profile:default-domain:default-project:netconf-default-profile controller/src/ifmap/ifmap_server_table.cc 95 2025-02-10 Mon 02:02:49:697.972 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459649259264, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating project-netconf-profile controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:697.986 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459649259264, Pid 109271]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating alarm:default-global-system-config:system-defined-process-status controller/src/ifmap/ifmap_server_table.cc 95 2025-02-10 Mon 02:02:49:698.006 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459649259264, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-alarm controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:698.021 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459649259264, Pid 109271]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating service-template:default-domain:haproxy-loadbalancer-template controller/src/ifmap/ifmap_server_table.cc 95 2025-02-10 Mon 02:02:49:698.042 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459649259264, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating domain-service-template controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:698.059 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459649259264, Pid 109271]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating job-template:default-global-system-config:image_upgrade_template controller/src/ifmap/ifmap_server_table.cc 95 2025-02-10 Mon 02:02:49:698.932 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459649259264, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-job-template controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:698.950 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459649259264, Pid 109271]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating service-template:default-domain:netns-snat-template controller/src/ifmap/ifmap_server_table.cc 95 2025-02-10 Mon 02:02:49:698.967 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459649259264, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating domain-service-template controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:698.982 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459649259264, Pid 109271]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating role-definition:default-global-system-config:dc-gateway-spine controller/src/ifmap/ifmap_server_table.cc 95 2025-02-10 Mon 02:02:49:698.998 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459649259264, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-role-definition controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:699.015 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459649259264, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:699.031 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459649259264, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:699.048 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459649259264, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:699.064 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459649259264, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:699.080 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459649259264, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:699.095 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459649259264, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:699.111 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459649259264, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:699.126 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459649259264, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:699.144 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459649259264, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-overlay-role controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:699.165 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459649259264, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-physical-role controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:699.185 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459649259264, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-role-definition controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:699.202 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459649259264, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:699.218 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459649259264, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:699.234 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459649259264, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:699.251 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459649259264, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-overlay-role controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:699.270 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459649259264, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-physical-role controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:699.292 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459649259264, Pid 109271]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating node-profile:default-global-system-config:device-functional-group controller/src/ifmap/ifmap_server_table.cc 95 2025-02-10 Mon 02:02:49:699.311 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459649259264, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-node-profile controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:700.441 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459649259264, Pid 109271]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating alarm:default-global-system-config:system-defined-storage-cluster-state controller/src/ifmap/ifmap_server_table.cc 95 2025-02-10 Mon 02:02:49:700.474 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459649259264, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-alarm controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:700.501 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459649259264, Pid 109271]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating telemetry-profile:default-domain:default-project:default-telemetry-profile-6 controller/src/ifmap/ifmap_server_table.cc 95 2025-02-10 Mon 02:02:49:700.521 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459649259264, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating project-telemetry-profile controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:700.542 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459649259264, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating telemetry-profile-netconf-profile controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:700.561 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459649259264, Pid 109271]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating role-definition:default-global-system-config:lean-superspine controller/src/ifmap/ifmap_server_table.cc 95 2025-02-10 Mon 02:02:49:700.578 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459649259264, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-role-definition controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:700.607 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459649259264, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:700.625 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459649259264, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:700.642 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459649259264, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:700.658 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459649259264, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-overlay-role controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:700.675 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459649259264, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-physical-role controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:700.689 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459649259264, Pid 109271]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating alarm:default-global-system-config:system-defined-node-status controller/src/ifmap/ifmap_server_table.cc 95 2025-02-10 Mon 02:02:49:700.706 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459649259264, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-alarm controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:700.726 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459649259264, Pid 109271]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating api-access-list:default-global-system-config:default-api-access-list controller/src/ifmap/ifmap_server_table.cc 95 2025-02-10 Mon 02:02:49:700.741 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459649259264, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-api-access-list controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:700.760 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459649259264, Pid 109271]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating role-config:default-global-system-config:juniper-qfx5120:basic controller/src/ifmap/ifmap_server_table.cc 95 2025-02-10 Mon 02:02:49:700.776 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459649259264, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating node-profile-role-config controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:700.790 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459649259264, Pid 109271]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating service-template:default-domain:analyzer-template controller/src/ifmap/ifmap_server_table.cc 95 2025-02-10 Mon 02:02:49:700.807 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459649259264, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating domain-service-template controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:700.822 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459649259264, Pid 109271]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating config-properties:default-global-system-config:config_property controller/src/ifmap/ifmap_server_table.cc 95 2025-02-10 Mon 02:02:49:700.839 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459649259264, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-config-properties controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:700.854 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459649259264, Pid 109271]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating job-template:default-global-system-config:device_deletion_template controller/src/ifmap/ifmap_server_table.cc 95 2025-02-10 Mon 02:02:49:700.893 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459649259264, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-job-template controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:700.914 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459649259264, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-role-definition controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:700.930 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459649259264, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:700.946 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459649259264, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:700.964 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459649259264, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:700.981 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459649259264, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:700.997 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459649259264, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:701.013 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459649259264, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:701.028 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459649259264, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:701.043 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459649259264, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:701.059 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459649259264, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:701.075 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459649259264, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-overlay-role controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:701.090 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459649259264, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-physical-role controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:701.110 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459649259264, Pid 109271]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating role-definition:default-global-system-config:crb-mcast-gateway-spine controller/src/ifmap/ifmap_server_table.cc 95 2025-02-10 Mon 02:02:49:701.125 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459649259264, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-role-definition controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:701.141 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459649259264, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:701.165 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459649259264, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:701.188 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459649259264, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:701.203 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459649259264, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:701.218 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459649259264, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:701.232 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459649259264, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:701.246 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459649259264, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:701.262 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459649259264, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:701.277 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459649259264, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-overlay-role controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:701.292 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459649259264, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-physical-role controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:701.306 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459649259264, Pid 109271]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating role-definition:default-global-system-config:rr-spine controller/src/ifmap/ifmap_server_table.cc 95 2025-02-10 Mon 02:02:49:701.324 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459649259264, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-role-definition controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:701.339 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459649259264, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:701.354 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459649259264, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:701.368 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459649259264, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:701.384 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459649259264, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:701.399 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459649259264, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-overlay-role controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:701.415 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459649259264, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-physical-role controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:701.429 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459649259264, Pid 109271]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating job-template:default-global-system-config:check_multicast_template controller/src/ifmap/ifmap_server_table.cc 95 2025-02-10 Mon 02:02:49:701.456 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459649259264, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-job-template controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:701.471 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459649259264, Pid 109271]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating job-template:default-global-system-config:show_ops_info_template controller/src/ifmap/ifmap_server_table.cc 95 2025-02-10 Mon 02:02:49:701.492 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459649259264, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-job-template controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:701.508 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459649259264, Pid 109271]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating alarm:default-global-system-config:system-defined-bottle-request-size-limit controller/src/ifmap/ifmap_server_table.cc 95 2025-02-10 Mon 02:02:49:701.524 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459649259264, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-alarm controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:701.537 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459649259264, Pid 109271]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating application-policy-set:default-policy-management:default-application-policy-set controller/src/ifmap/ifmap_server_table.cc 95 2025-02-10 Mon 02:02:49:701.555 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459649259264, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating policy-management-application-policy-set controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:701.574 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459649259264, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating application-policy-set-global-vrouter-config controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:701.588 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459649259264, Pid 109271]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating service-template:default-domain:docker-template controller/src/ifmap/ifmap_server_table.cc 95 2025-02-10 Mon 02:02:49:701.604 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459649259264, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating domain-service-template controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:701.618 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459649259264, Pid 109271]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating analytics-alarm-node:default-global-system-config:cn-jenkins-deploy-platform-ansible-os-2705-2. controller/src/ifmap/ifmap_server_table.cc 95 2025-02-10 Mon 02:02:49:701.636 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459649259264, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-analytics-alarm-node controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:701.649 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459649259264, Pid 109271]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating device-functional-group:default-domain:default-project:Centrally-Routed-Border-Spine controller/src/ifmap/ifmap_server_table.cc 95 2025-02-10 Mon 02:02:49:701.667 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459649259264, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating project-device-functional-group controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:701.689 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459649259264, Pid 109271]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating device-functional-group-physical-role controller/src/ifmap/ifmap_server_table.cc 130 2025-02-10 Mon 02:02:49:709.191 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140460034267264, Pid 109271]: DnsAgent [SYS_NOTICE]: DnsOperational: EndOfRib point was reached, IF-MAP objects have been created controller/src/dns/main.cc 86 2025-02-10 Mon 02:02:49:709.230 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140460034267264, Pid 109271]: DnsAgent [SYS_NOTICE]: DnsOperational: The IF-MAP database table has been updated: __ifmap__.virtual_DNS.0, counting 0 elements controller/src/dns/main.cc 98 2025-02-10 Mon 02:02:49:709.242 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140460034267264, Pid 109271]: DnsAgent [SYS_NOTICE]: DnsOperational: The IF-MAP database table has been updated: __ifmap__.virtual_DNS_record.0, counting 0 elements controller/src/dns/main.cc 98 2025-02-10 Mon 02:02:49:709.253 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140460034267264, Pid 109271]: DnsAgent [SYS_NOTICE]: DnsOperational: The IF-MAP database table has been updated: __ifmap__.network_ipam.0, counting 3 elements controller/src/dns/main.cc 98 2025-02-10 Mon 02:02:49:709.262 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140460034267264, Pid 109271]: DnsAgent [SYS_NOTICE]: DnsOperational: The IF-MAP database table has been updated: __ifmap__.virtual_network_network_ipam.0, counting 1 elements controller/src/dns/main.cc 98 2025-02-10 Mon 02:02:49:709.272 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140460034267264, Pid 109271]: DnsAgent [SYS_NOTICE]: DnsOperational: The IF-MAP database table has been updated: __ifmap__.global_qos_config.0, counting 1 elements controller/src/dns/main.cc 98 2025-02-10 Mon 02:02:49:709.280 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140460034267264, Pid 109271]: DnsAgent [SYS_NOTICE]: DnsOperational: VnniConfig has been updated after the bulk sync, the number of elements: 1 controller/src/dns/main.cc 109 2025-02-10 Mon 02:02:49:709.289 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140460034267264, Pid 109271]: DnsAgent [SYS_NOTICE]: DnsOperational: IpamConfig has been updated after the bulk sync, the number of elements: 3 controller/src/dns/main.cc 120 2025-02-10 Mon 02:02:49:709.297 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140460034267264, Pid 109271]: DnsAgent [SYS_NOTICE]: DnsOperational: VirtualDnsConfig has been updated after the bulk sync, the number of elements: 0 controller/src/dns/main.cc 131 2025-02-10 Mon 02:02:49:709.314 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140460034267264, Pid 109271]: DnsAgent [SYS_NOTICE]: DnsOperational: VirtualDnsRecordConfig has been updated after the bulk sync, the number of elements: 0 controller/src/dns/main.cc 142 2025-02-10 Mon 02:02:49:709.322 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140460034267264, Pid 109271]: DnsAgent [SYS_NOTICE]: DnsOperational: GlobalQosConfig has been updated after the bulk sync controller/src/dns/main.cc 152 2025-02-10 Mon 02:02:49:709.387 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140460034267264, Pid 109271]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: RabbitMQ SM: RabbitMQ connection down src/contrail-common/config-client-mgr/config_amqp_client.cc 158 2025-02-10 Mon 02:02:49:709.399 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140460034267264, Pid 109271]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: RabbitMQ SM: Connect to Rabbit MQ with queue_delete TRUE src/contrail-common/config-client-mgr/config_amqp_client.cc 167 2025-02-10 Mon 02:02:49:709.521 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459649259264, Pid 109271]: Processing scm::EvSandeshSend in state Connect 2025-02-10 Mon 02:02:49:709.561 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459649259264, Pid 109271]: SANDESH: Send FAILED: 1739152969709357 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-2705-3 process_status= [ [ [ module_id = contrail-dns instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.17:8086, ] ] status = Initializing description = Disconnect : EvCollectorUpdate -> Connect ], [ type = Database name = Cassandra server_addrs= [ [ (*_iter6) = 10.0.0.17:9041, (*_iter6) = 10.0.0.12:9041, (*_iter6) = 10.0.0.30:9041, ] ] status = Up description = Established Cassandra connection ], [ type = Database name = RabbitMQ server_addrs= [ [ (*_iter6) = 10.0.0.17:5673, (*_iter6) = 10.0.0.12:5673, (*_iter6) = 10.0.0.30:5673, ] ] status = Down description = RabbitMQ connection down ], ] ] flag_infos= [ [ ] ] description = Collector, Database:RabbitMQ connection down ], ] ] ] 2025-02-10 Mon 02:02:49:709.577 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459649259264, Pid 109271]: Wrong state: Connect for event: EvSandeshSend message: NodeStatusUVE 2025-02-10 Mon 02:02:49:779.581 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459632465664, Pid 109271]: Processing scm::EvSandeshSend in state Connect 2025-02-10 Mon 02:02:49:779.755 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459632465664, Pid 109271]: SANDESH: Send FAILED: 1739152969779518 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-2705-3 process_status= [ [ [ module_id = contrail-dns instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.17:8086, ] ] status = Initializing description = Disconnect : EvCollectorUpdate -> Connect ], [ type = Database name = Cassandra server_addrs= [ [ (*_iter6) = 10.0.0.17:9041, (*_iter6) = 10.0.0.12:9041, (*_iter6) = 10.0.0.30:9041, ] ] status = Up description = Established Cassandra connection ], [ type = Database name = RabbitMQ server_addrs= [ [ (*_iter6) = 10.0.0.17:5673, (*_iter6) = 10.0.0.12:5673, (*_iter6) = 10.0.0.30:5673, ] ] status = Up description = RabbitMQ connection established ], ] ] flag_infos= [ [ ] ] description = Collector connection down ], ] ] ] 2025-02-10 Mon 02:02:49:779.766 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459632465664, Pid 109271]: Wrong state: Connect for event: EvSandeshSend message: NodeStatusUVE 2025-02-10 Mon 02:02:49:780.028 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140460034267264, Pid 109271]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: RabbitMQ SM: RabbitMQ connection established src/contrail-common/config-client-mgr/config_amqp_client.cc 151 2025-02-10 Mon 02:02:49:780.084 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459640862464, Pid 109271]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: TRUE src/contrail-common/config-client-mgr/config_client_manager.cc 225 2025-02-10 Mon 02:02:49:783.853 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140460034267264, Pid 109271]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.0.0.30:45698::10.0.0.17:8086(-1) < Active session connection complete src/contrail-common/io/tcp_session.cc 282 2025-02-10 Mon 02:02:49:783.900 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140460034267264, Pid 109271]: 10.0.0.30:45698::10.0.0.17:8086(-1) OnSessionEvent TCP Connected 2025-02-10 Mon 02:02:49:783.916 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140460034267264, Pid 109271]: 10.0.0.30:45698::10.0.0.17:8086(-1):EvTcpConnected 2025-02-10 Mon 02:02:49:784.075 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459657656064, Pid 109271]: DnsAgent [SYS_NOTICE]: DnsOperational: BIND named up; DNS is operational controller/src/dns/mgr/dns_mgr.cc 737 2025-02-10 Mon 02:02:49:784.106 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459657656064, Pid 109271]: Processing scm::EvTcpConnected in state Connect 2025-02-10 Mon 02:02:49:784.116 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459657656064, Pid 109271]: Connect : EvTcpConnected : Cancelling Connect timer 2025-02-10 Mon 02:02:49:784.152 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459657656064, Pid 109271]: ClientInit 2025-02-10 Mon 02:02:49:784.189 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459657656064, Pid 109271]: Sending Ctrl Message for cn-jenkins-deploy-platform-ansible-os-2705-3:contrail-dns:0:Control count 1 2025-02-10 Mon 02:02:49:784.310 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459657656064, Pid 109271]: Processing scm::EvSandeshSend in state ClientInit 2025-02-10 Mon 02:02:49:784.320 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459657656064, Pid 109271]: ClientInit : EvSandeshSend : NodeStatusUVE 2025-02-10 Mon 02:02:49:784.344 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459657656064, Pid 109271]: SANDESH: Send FAILED: 1739152969784143 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-2705-3 process_status= [ [ [ module_id = contrail-dns instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.17:8086, ] ] status = Initializing description = Connect : EvTcpConnected ], [ type = Database name = Cassandra server_addrs= [ [ (*_iter6) = 10.0.0.17:9041, (*_iter6) = 10.0.0.12:9041, (*_iter6) = 10.0.0.30:9041, ] ] status = Up description = Established Cassandra connection ], [ type = Database name = RabbitMQ server_addrs= [ [ (*_iter6) = 10.0.0.17:5673, (*_iter6) = 10.0.0.12:5673, (*_iter6) = 10.0.0.30:5673, ] ] status = Up description = RabbitMQ connection established ], ] ] flag_infos= [ [ ] ] description = Collector connection down ], ] ] ] 2025-02-10 Mon 02:02:49:784.351 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459657656064, Pid 109271]: Received UVE message in wrong state : NodeStatusUVE 2025-02-10 Mon 02:02:49:784.359 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459657656064, Pid 109271]: Processing scm::EvSandeshSend in state ClientInit 2025-02-10 Mon 02:02:49:784.366 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459657656064, Pid 109271]: ClientInit : EvSandeshSend : SandeshCtrlClientToServer 2025-02-10 Mon 02:02:49:784.389 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459657656064, Pid 109271]: Processing scm::EvSandeshSend in state ClientInit 2025-02-10 Mon 02:02:49:784.396 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459657656064, Pid 109271]: ClientInit : EvSandeshSend : SandeshModuleClientTrace 2025-02-10 Mon 02:02:49:784.548 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459657656064, Pid 109271]: SANDESH: Send FAILED: 1739152969784297 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-2705-3:Control:contrail-dns:0 client_info= [ status = ClientInit successful_connections = 1 pid = 109271 http_port = 8092 start_time = 1739152969077724 collector_name = collector_ip = 10.0.0.17:8086 collector_list= [ [ (*_iter40) = 10.0.0.17:8086, (*_iter40) = 10.0.0.30:8086, (*_iter40) = 10.0.0.12:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ConfigClientMgrDebug [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 152 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigurationMalformedReferenceWarningLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 14 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = DnsOperational [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 13 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapLinkOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 469 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapNodeOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 271 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 7 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 3 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 922, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 10, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-02-10 Mon 02:02:49:784.567 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459657656064, Pid 109271]: Received UVE message in wrong state : SandeshModuleClientTrace 2025-02-10 Mon 02:02:49:786.326 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459653457664, Pid 109271]: OnMessage control in state: ClientInit 2025-02-10 Mon 02:02:49:786.373 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459653457664, Pid 109271]: Processing scm::EvSandeshMessageRecv in state ClientInit 2025-02-10 Mon 02:02:49:786.382 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459653457664, Pid 109271]: ClientInit : EvSandeshMessageRecv 2025-02-10 Mon 02:02:49:786.405 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459653457664, Pid 109271]: Received Ctrl Message with size 0 2025-02-10 Mon 02:02:49:786.415 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459653457664, Pid 109271]: SyncAllMaps for DnsState:0 period 0 cycle 0 without seqno , total = 0 2025-02-10 Mon 02:02:49:786.587 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459653457664, Pid 109271]: SyncUVE Syncing ObjectGeneratorInfo val name = cn-jenkins-deploy-platform-ansible-os-2705-3:Control:contrail-dns:0 client_info= [ status = ClientInit successful_connections = 1 pid = 109271 http_port = 8092 start_time = 1739152969077724 collector_name = collector_ip = 10.0.0.17:8086 collector_list= [ [ (*_iter40) = 10.0.0.17:8086, (*_iter40) = 10.0.0.30:8086, (*_iter40) = 10.0.0.12:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ConfigClientMgrDebug [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 152 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigurationMalformedReferenceWarningLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 14 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = DnsOperational [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 13 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapLinkOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 469 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapNodeOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 271 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 7 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 3 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 922, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 10, _iter59->first = sent _iter59->second = 0, ] ] proxy seq 5 2025-02-10 Mon 02:02:49:786.613 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459653457664, Pid 109271]: SyncAllMaps for ModuleClientState:0 period 0 cycle 0 without seqno , total = 1 2025-02-10 Mon 02:02:49:786.636 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459653457664, Pid 109271]: SyncUVE Syncing ObjectDns val name = cn-jenkins-deploy-platform-ansible-os-2705-3 process_status= [ [ [ module_id = contrail-dns instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.17:8086, ] ] status = Initializing description = Connect : EvTcpConnected ], [ type = Database name = Cassandra server_addrs= [ [ (*_iter6) = 10.0.0.17:9041, (*_iter6) = 10.0.0.12:9041, (*_iter6) = 10.0.0.30:9041, ] ] status = Up description = Established Cassandra connection ], [ type = Database name = RabbitMQ server_addrs= [ [ (*_iter6) = 10.0.0.17:5673, (*_iter6) = 10.0.0.12:5673, (*_iter6) = 10.0.0.30:5673, ] ] status = Up description = RabbitMQ connection established ], ] ] flag_infos= [ [ ] ] description = Collector connection down ], ] ] proxy seq 8 2025-02-10 Mon 02:02:49:786.648 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459653457664, Pid 109271]: SyncAllMaps for NodeStatus:0 period 0 cycle 0 without seqno , total = 1 2025-02-10 Mon 02:02:49:786.659 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459653457664, Pid 109271]: SyncAllMaps for PeerFlapData:0 period 0 cycle 0 without seqno , total = 0 2025-02-10 Mon 02:02:49:786.666 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459653457664, Pid 109271]: SyncAllMaps for PeerStatsData:0 period 0 cycle 0 without seqno , total = 0 2025-02-10 Mon 02:02:49:786.675 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459653457664, Pid 109271]: SyncAllMaps for XmppPeerInfoData:0 period 0 cycle 0 without seqno , total = 0 2025-02-10 Mon 02:02:49:786.706 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459653457664, Pid 109271]: Established 2025-02-10 Mon 02:02:50:831.914 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459636664064, Pid 109271]: Applying named configuration failed 2025-02-10 Mon 02:02:50:831.998 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459636664064, Pid 109271]: DnsAgent [SYS_NOTICE]: DnsOperational: Bulk sync has been done (the DB has been synced with named) controller/src/dns/mgr/dns_mgr.cc 455 2025-02-10 Mon 02:02:51:585.457 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459636664064, Pid 109271]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-02-10 Mon 02:03:15:962.267 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459640862464, Pid 109271]: ConfigClientManager [SYS_WARN]: ConfigClientMgrWarning: RabbitMQ SM: Caught exception while receiving messages from RabbitMQ: 10.0.0.17:5673 : connection error: 320: CONNECTION_FORCED - broker forced connection closure with reason 'shutdown' src/contrail-common/config-client-mgr/config_amqp_client.cc 366 2025-02-10 Mon 02:03:15:965.604 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459640862464, Pid 109271]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: RabbitMQ SM: RabbitMQ connection down src/contrail-common/config-client-mgr/config_amqp_client.cc 158 2025-02-10 Mon 02:03:15:965.644 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459640862464, Pid 109271]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: RabbitMQ SM: Connect to Rabbit MQ with queue_delete FALSE src/contrail-common/config-client-mgr/config_amqp_client.cc 167 2025-02-10 Mon 02:03:15:969.341 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459640862464, Pid 109271]: ConfigClientManager [SYS_WARN]: ConfigClientMgrWarning: RabbitMQ SM: Caught exception while connecting to RabbitMQ: 10.0.0.17:5673 : a socket error occurred src/contrail-common/config-client-mgr/config_amqp_client.cc 220 2025-02-10 Mon 02:03:16:308.194 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459640862464, Pid 109271]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: RabbitMQ SM: RabbitMQ connection established src/contrail-common/config-client-mgr/config_amqp_client.cc 151 2025-02-10 Mon 02:04:44:533.310 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459640862464, Pid 109271]: ConfigClientManager [SYS_WARN]: ConfigClientMgrWarning: RabbitMQ SM: Caught exception while receiving messages from RabbitMQ: 10.0.0.12:5673 : connection error: 320: CONNECTION_FORCED - broker forced connection closure with reason 'shutdown' src/contrail-common/config-client-mgr/config_amqp_client.cc 366 2025-02-10 Mon 02:04:44:535.306 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459640862464, Pid 109271]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: RabbitMQ SM: RabbitMQ connection down src/contrail-common/config-client-mgr/config_amqp_client.cc 158 2025-02-10 Mon 02:04:44:535.346 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459640862464, Pid 109271]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: RabbitMQ SM: Connect to Rabbit MQ with queue_delete FALSE src/contrail-common/config-client-mgr/config_amqp_client.cc 167 2025-02-10 Mon 02:04:44:536.574 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459640862464, Pid 109271]: ConfigClientManager [SYS_WARN]: ConfigClientMgrWarning: RabbitMQ SM: Caught exception while connecting to RabbitMQ: 10.0.0.12:5673 : a socket error occurred src/contrail-common/config-client-mgr/config_amqp_client.cc 220 2025-02-10 Mon 02:04:44:605.988 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459640862464, Pid 109271]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: RabbitMQ SM: RabbitMQ connection established src/contrail-common/config-client-mgr/config_amqp_client.cc 151 2025-02-10 Mon 02:05:08:340.640 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140460034267264, Pid 109271]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.20.0.8:8092::10.20.0.6:44858 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 256 2025-02-10 Mon 02:05:09:844.699 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140460034267264, Pid 109271]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.20.0.8:8092::10.20.0.8:35350 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 256 2025-02-10 Mon 02:05:09:868.975 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140460034267264, Pid 109271]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.20.0.8:8092::10.20.0.11:45824 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 256 2025-02-10 Mon 02:05:35:047.502 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140460034267264, Pid 109271]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.20.0.8:8092::10.20.0.8:48106 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 256 2025-02-10 Mon 02:05:35:609.341 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140460034267264, Pid 109271]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.20.0.8:8092::10.20.0.6:41936 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 256 2025-02-10 Mon 02:05:36:470.394 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140460034267264, Pid 109271]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.20.0.8:8092::10.20.0.11:38706 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 256 2025-02-10 Mon 02:06:16:313.468 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459640862464, Pid 109271]: ConfigClientManager [SYS_WARN]: ConfigClientMgrWarning: RabbitMQ SM: Caught exception while receiving messages from RabbitMQ: 10.0.0.30:5673 : connection error: 320: CONNECTION_FORCED - broker forced connection closure with reason 'shutdown' src/contrail-common/config-client-mgr/config_amqp_client.cc 366 2025-02-10 Mon 02:06:16:313.647 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459640862464, Pid 109271]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: RabbitMQ SM: RabbitMQ connection down src/contrail-common/config-client-mgr/config_amqp_client.cc 158 2025-02-10 Mon 02:06:16:313.667 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459640862464, Pid 109271]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: RabbitMQ SM: Connect to Rabbit MQ with queue_delete FALSE src/contrail-common/config-client-mgr/config_amqp_client.cc 167 2025-02-10 Mon 02:06:16:313.907 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459640862464, Pid 109271]: ConfigClientManager [SYS_WARN]: ConfigClientMgrWarning: RabbitMQ SM: Caught exception while connecting to RabbitMQ: 10.0.0.30:5673 : a socket error occurred src/contrail-common/config-client-mgr/config_amqp_client.cc 220 2025-02-10 Mon 02:06:16:397.309 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459640862464, Pid 109271]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: RabbitMQ SM: RabbitMQ connection established src/contrail-common/config-client-mgr/config_amqp_client.cc 151 2025-02-10 Mon 02:10:08:345.549 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140460034267264, Pid 109271]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.20.0.8:8092::10.20.0.6:53784 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 256 2025-02-10 Mon 02:10:09:850.069 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140460034267264, Pid 109271]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.20.0.8:8092::10.20.0.8:49938 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 256 2025-02-10 Mon 02:10:09:878.984 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140460034267264, Pid 109271]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.20.0.8:8092::10.20.0.11:43712 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 256 2025-02-10 Mon 02:10:35:047.816 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140460034267264, Pid 109271]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.20.0.8:8092::10.20.0.8:50102 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 256 2025-02-10 Mon 02:10:35:607.389 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140460034267264, Pid 109271]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.20.0.8:8092::10.20.0.6:39704 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 256 2025-02-10 Mon 02:10:36:473.295 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140460034267264, Pid 109271]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.20.0.8:8092::10.20.0.11:38196 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 256 2025-02-10 Mon 02:15:08:346.051 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140460034267264, Pid 109271]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.20.0.8:8092::10.20.0.6:32996 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 256 2025-02-10 Mon 02:15:09:853.033 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140460034267264, Pid 109271]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.20.0.8:8092::10.20.0.8:33346 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 256 2025-02-10 Mon 02:15:09:882.443 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140460034267264, Pid 109271]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.20.0.8:8092::10.20.0.11:35904 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 256 2025-02-10 Mon 02:15:35:052.456 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140460034267264, Pid 109271]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.20.0.8:8092::10.20.0.8:51282 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 256 2025-02-10 Mon 02:15:35:610.737 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140460034267264, Pid 109271]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.20.0.8:8092::10.20.0.6:37580 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 256 2025-02-10 Mon 02:15:36:476.766 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140460034267264, Pid 109271]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.20.0.8:8092::10.20.0.11:34622 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 256 2025-02-10 Mon 02:19:40:661.984 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459867805440, Pid 109271]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-02-10 Mon 02:19:40:721.248 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459624068864, Pid 109271]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-02-10 Mon 02:19:40:900.352 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459872003840, Pid 109271]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-02-10 Mon 02:19:40:958.925 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140459657656064, Pid 109271]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-02-10 Mon 02:20:08:350.605 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140460034267264, Pid 109271]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.20.0.8:8092::10.20.0.6:39866 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 256 2025-02-10 Mon 02:20:09:854.647 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140460034267264, Pid 109271]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.20.0.8:8092::10.20.0.8:52088 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 256 2025-02-10 Mon 02:20:09:883.532 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140460034267264, Pid 109271]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.20.0.8:8092::10.20.0.11:50608 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 256 2025-02-10 Mon 02:20:35:055.066 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140460034267264, Pid 109271]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.20.0.8:8092::10.20.0.8:48526 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 256 2025-02-10 Mon 02:20:35:613.979 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140460034267264, Pid 109271]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.20.0.8:8092::10.20.0.6:57232 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 256 2025-02-10 Mon 02:20:36:478.589 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140460034267264, Pid 109271]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.20.0.8:8092::10.20.0.11:40166 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 256 2025-02-10 Mon 02:25:08:356.243 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140460034267264, Pid 109271]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.20.0.8:8092::10.20.0.6:41602 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 256 2025-02-10 Mon 02:25:09:860.874 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140460034267264, Pid 109271]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.20.0.8:8092::10.20.0.8:38608 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 256 2025-02-10 Mon 02:25:09:889.795 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140460034267264, Pid 109271]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.20.0.8:8092::10.20.0.11:56816 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 256 2025-02-10 Mon 02:25:35:054.728 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140460034267264, Pid 109271]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.20.0.8:8092::10.20.0.8:48414 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 256 2025-02-10 Mon 02:25:35:614.190 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140460034267264, Pid 109271]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.20.0.8:8092::10.20.0.6:45694 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 256 2025-02-10 Mon 02:25:36:482.155 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140460034267264, Pid 109271]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.20.0.8:8092::10.20.0.11:52016 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 256 2025-02-10 Mon 02:30:08:358.185 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140460034267264, Pid 109271]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.20.0.8:8092::10.20.0.6:57348 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 256 2025-02-10 Mon 02:30:09:862.604 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140460034267264, Pid 109271]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.20.0.8:8092::10.20.0.8:58666 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 256 2025-02-10 Mon 02:30:09:890.743 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140460034267264, Pid 109271]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.20.0.8:8092::10.20.0.11:33342 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 256 2025-02-10 Mon 02:30:35:057.528 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140460034267264, Pid 109271]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.20.0.8:8092::10.20.0.8:55142 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 256 2025-02-10 Mon 02:30:35:614.999 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140460034267264, Pid 109271]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.20.0.8:8092::10.20.0.6:48732 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 256 2025-02-10 Mon 02:30:36:482.243 UTC cn-jenkins-deploy-platform-ansible-os-2705-3 [Thread 140460034267264, Pid 109271]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.20.0.8:8092::10.20.0.11:36294 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 256