2025-01-20 Mon 01:50:15:024.344 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502768416512, Pid 77462]: Wrong state: Connect for event: EvSandeshSend message: SandeshModuleClientTrace 2025-01-20 Mon 01:50:15:024.376 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502768416512, Pid 77462]: Processing scm::EvSandeshSend in state Connect 2025-01-20 Mon 01:50:15:024.423 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502768416512, Pid 77462]: SANDESH: Send FAILED: 1737337815019028 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-2650-2 process_status= [ [ [ module_id = contrail-dns instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.42:8086, ] ] status = Initializing description = Established : EvTcpClose -> Connect ], [ type = Database name = Cassandra server_addrs= [ [ (*_iter6) = 10.0.0.42:9041, (*_iter6) = 10.0.0.22:9041, (*_iter6) = 10.0.0.49:9041, ] ] status = Up description = Established Cassandra connection ], [ type = Database name = RabbitMQ server_addrs= [ [ (*_iter6) = 10.0.0.42:5673, (*_iter6) = 10.0.0.22:5673, (*_iter6) = 10.0.0.49:5673, ] ] status = Up description = RabbitMQ connection established ], ] ] flag_infos= [ [ ] ] description = Collector connection down ], ] ] ] 2025-01-20 Mon 01:50:15:024.431 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502768416512, Pid 77462]: Wrong state: Connect for event: EvSandeshSend message: NodeStatusUVE 2025-01-20 Mon 01:50:15:024.441 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502768416512, Pid 77462]: Processing scm::EvTcpDeleteSession in state Connect 2025-01-20 Mon 01:50:15:024.481 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502768416512, Pid 77462]: Processing scm::EvSandeshSend in state Connect 2025-01-20 Mon 01:50:15:024.781 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502768416512, Pid 77462]: SANDESH: Send FAILED: 1737337815019252 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-2650-2:Control:contrail-dns:0 client_info= [ status = Connect successful_connections = 1 pid = 77462 http_port = 8092 start_time = 1737337641318269 collector_name = collector_ip = 10.0.0.49:8086 collector_list= [ [ (*_iter40) = 10.0.0.42:8086, (*_iter40) = 10.0.0.49:8086, (*_iter40) = 10.0.0.22:8086, ] ] rx_socket_stats= [ bytes = 1017 calls = 2 average_bytes = 508 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 218906 calls = 18 average_bytes = 12161 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ConfigClientGetRowError [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 6 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigClientMgrDebug [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 648 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigurationMalformedReferenceWarningLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 17 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = DnsOperational [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 14 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapLinkOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 469 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapNodeOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 271 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 3 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 10 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlClientToServer [ messages_sent = 1 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlServerToClient [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 10 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 6 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 9 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveDnsInfo [ messages_sent = 1 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionCreate [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 6 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppCreateConnection [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 1442, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 16, _iter59->first = sent _iter59->second = 21, ] ] ] 2025-01-20 Mon 01:50:15:024.811 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502768416512, Pid 77462]: Wrong state: Connect for event: EvSandeshSend message: SandeshModuleClientTrace 2025-01-20 Mon 01:50:15:028.202 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502953732224, Pid 77462]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.0.0.22:40548::10.0.0.49:8086(-1) < Active session connection complete src/contrail-common/io/tcp_session.cc 280 2025-01-20 Mon 01:50:15:028.238 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502953732224, Pid 77462]: 10.0.0.22:40548::10.0.0.49:8086(-1) OnSessionEvent TCP Connected 2025-01-20 Mon 01:50:15:028.248 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502953732224, Pid 77462]: 10.0.0.22:40548::10.0.0.49:8086(-1):EvTcpConnected 2025-01-20 Mon 01:50:15:031.897 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502797805312, Pid 77462]: Processing scm::EvTcpConnected in state Connect 2025-01-20 Mon 01:50:15:031.939 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502797805312, Pid 77462]: Connect : EvTcpConnected : Cancelling Connect timer 2025-01-20 Mon 01:50:15:031.995 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502797805312, Pid 77462]: ClientInit 2025-01-20 Mon 01:50:15:032.026 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502797805312, Pid 77462]: Sending Ctrl Message for cn-jenkins-deploy-platform-ansible-os-2650-2:contrail-dns:0:Control count 2 2025-01-20 Mon 01:50:15:032.208 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502797805312, Pid 77462]: Processing scm::EvSandeshSend in state ClientInit 2025-01-20 Mon 01:50:15:032.221 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502797805312, Pid 77462]: ClientInit : EvSandeshSend : NodeStatusUVE 2025-01-20 Mon 01:50:15:032.246 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502797805312, Pid 77462]: SANDESH: Send FAILED: 1737337815031982 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-2650-2 process_status= [ [ [ module_id = contrail-dns instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.49:8086, ] ] status = Initializing description = Connect : EvTcpConnected ], [ type = Database name = Cassandra server_addrs= [ [ (*_iter6) = 10.0.0.42:9041, (*_iter6) = 10.0.0.22:9041, (*_iter6) = 10.0.0.49:9041, ] ] status = Up description = Established Cassandra connection ], [ type = Database name = RabbitMQ server_addrs= [ [ (*_iter6) = 10.0.0.42:5673, (*_iter6) = 10.0.0.22:5673, (*_iter6) = 10.0.0.49:5673, ] ] status = Up description = RabbitMQ connection established ], ] ] flag_infos= [ [ ] ] description = Collector connection down ], ] ] ] 2025-01-20 Mon 01:50:15:032.263 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502797805312, Pid 77462]: Received UVE message in wrong state : NodeStatusUVE 2025-01-20 Mon 01:50:15:032.272 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502797805312, Pid 77462]: Processing scm::EvSandeshSend in state ClientInit 2025-01-20 Mon 01:50:15:032.278 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502797805312, Pid 77462]: ClientInit : EvSandeshSend : SandeshCtrlClientToServer 2025-01-20 Mon 01:50:15:032.289 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502797805312, Pid 77462]: Processing scm::EvSandeshSend in state ClientInit 2025-01-20 Mon 01:50:15:032.295 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502797805312, Pid 77462]: ClientInit : EvSandeshSend : SandeshModuleClientTrace 2025-01-20 Mon 01:50:15:032.544 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502797805312, Pid 77462]: SANDESH: Send FAILED: 1737337815032191 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-2650-2:Control:contrail-dns:0 client_info= [ status = ClientInit successful_connections = 2 pid = 77462 http_port = 8092 start_time = 1737337641318269 collector_name = collector_ip = 10.0.0.49:8086 collector_list= [ [ (*_iter40) = 10.0.0.42:8086, (*_iter40) = 10.0.0.49:8086, (*_iter40) = 10.0.0.22:8086, ] ] rx_socket_stats= [ bytes = 1017 calls = 2 average_bytes = 508 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 218906 calls = 18 average_bytes = 12161 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ConfigClientGetRowError [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 6 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigClientMgrDebug [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 648 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigurationMalformedReferenceWarningLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 17 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = DnsOperational [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 14 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapLinkOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 469 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapNodeOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 271 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 3 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 11 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlClientToServer [ messages_sent = 1 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlServerToClient [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 10 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 8 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 10 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveDnsInfo [ messages_sent = 1 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionCreate [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 6 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppCreateConnection [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 1443, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 19, _iter59->first = sent _iter59->second = 21, ] ] ] 2025-01-20 Mon 01:50:15:032.568 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502797805312, Pid 77462]: Received UVE message in wrong state : SandeshModuleClientTrace 2025-01-20 Mon 01:50:15:071.495 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502797805312, Pid 77462]: OnMessage control in state: ClientInit 2025-01-20 Mon 01:50:15:071.562 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502764218112, Pid 77462]: Processing scm::EvSandeshMessageRecv in state ClientInit 2025-01-20 Mon 01:50:15:071.590 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502764218112, Pid 77462]: ClientInit : EvSandeshMessageRecv 2025-01-20 Mon 01:50:15:071.622 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502764218112, Pid 77462]: Received Ctrl Message with size 0 2025-01-20 Mon 01:50:15:071.643 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502764218112, Pid 77462]: SyncUVE Syncing ObjectDns val name = cn-jenkins-deploy-platform-ansible-os-2650-2 start_time = 1737337641884778 build_info = {"build-info":[{"build-version":"nightly.r24.1.centos","build-time":"2025-01-16 00:14:43.877305","build-user":"root","build-hostname":"jenkins-build-prepare-sandbox-7545-1.novalocal","build-id":"nightly.r24.1.centos-nightly_r24.1_centos.el7@contrail","build-number":"unknown"}]} collector = 10.0.0.42:8086 self_ip_list= [ [ (*_iter12) = 10.20.0.11, ] ] proxy seq 1 2025-01-20 Mon 01:50:15:071.656 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502764218112, Pid 77462]: SyncAllMaps for DnsState:0 period 0 cycle 0 without seqno , total = 1 2025-01-20 Mon 01:50:15:071.990 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502764218112, Pid 77462]: SyncUVE Syncing ObjectGeneratorInfo val name = cn-jenkins-deploy-platform-ansible-os-2650-2:Control:contrail-dns:0 client_info= [ status = ClientInit successful_connections = 2 pid = 77462 http_port = 8092 start_time = 1737337641318269 collector_name = collector_ip = 10.0.0.49:8086 collector_list= [ [ (*_iter40) = 10.0.0.42:8086, (*_iter40) = 10.0.0.49:8086, (*_iter40) = 10.0.0.22:8086, ] ] rx_socket_stats= [ bytes = 1017 calls = 2 average_bytes = 508 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 218906 calls = 18 average_bytes = 12161 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] sm_queue_count = 0 sm_stats= [ ev_stats= [ [ [ event = scm::EvCollectorUpdate enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvSandeshMessageRecv enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvSandeshSend enqueues = 35 dequeues = 35 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvStart enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpConnected enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = enqueues = 39 dequeues = 39 enqueue_fails = 0 dequeue_fails = 0 ], ] ] state = Established state_since = 1737337674608622 last_state = Established last_event = scm::EvSandeshSend last_event_at = 1737337764540176 ] session_stats= [ num_recv_msg = 0 num_recv_msg_fail = 0 num_recv_fail = 0 num_send_msg = 19 num_send_msg_fail = 0 num_send_buffer_fail = 0 num_wait_msgq_enqueue = 0 num_wait_msgq_dequeue = 0 num_write_ready_cb_error = 0 ] session_rx_socket_stats= [ bytes = 1017 calls = 2 average_bytes = 508 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] session_tx_socket_stats= [ bytes = 183601 calls = 16 average_bytes = 11475 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] msg_type_agg= [ [ _iter58->first = ConfigClientGetRowError [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 6 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigClientMgrDebug [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 648 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigurationMalformedReferenceWarningLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 17 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = DnsOperational [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 14 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapLinkOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 469 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapNodeOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 271 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 3 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 11 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlClientToServer [ messages_sent = 1 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlServerToClient [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 10 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 8 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 10 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveDnsInfo [ messages_sent = 1 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionCreate [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 6 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppCreateConnection [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 1443, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 19, _iter59->first = sent _iter59->second = 21, ] ] max_sm_queue_count = 7 proxy seq 19 2025-01-20 Mon 01:50:15:072.030 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502764218112, Pid 77462]: SyncAllMaps for ModuleClientState:0 period 0 cycle 0 without seqno , total = 1 2025-01-20 Mon 01:50:15:072.055 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502764218112, Pid 77462]: SyncUVE Syncing ObjectDns val name = cn-jenkins-deploy-platform-ansible-os-2650-2 process_status= [ [ [ module_id = contrail-dns instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.49:8086, ] ] status = Initializing description = Connect : EvTcpConnected ], [ type = Database name = Cassandra server_addrs= [ [ (*_iter6) = 10.0.0.42:9041, (*_iter6) = 10.0.0.22:9041, (*_iter6) = 10.0.0.49:9041, ] ] status = Up description = Established Cassandra connection ], [ type = Database name = RabbitMQ server_addrs= [ [ (*_iter6) = 10.0.0.42:5673, (*_iter6) = 10.0.0.22:5673, (*_iter6) = 10.0.0.49:5673, ] ] status = Up description = RabbitMQ connection established ], ] ] flag_infos= [ [ ] ] description = Collector connection down ], ] ] proxy seq 14 2025-01-20 Mon 01:50:15:072.070 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502764218112, Pid 77462]: SyncAllMaps for NodeStatus:0 period 0 cycle 0 without seqno , total = 1 2025-01-20 Mon 01:50:15:072.081 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502764218112, Pid 77462]: SyncAllMaps for PeerFlapData:0 period 0 cycle 0 without seqno , total = 0 2025-01-20 Mon 01:50:15:072.099 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502764218112, Pid 77462]: SyncAllMaps for PeerStatsData:0 period 0 cycle 0 without seqno , total = 0 2025-01-20 Mon 01:50:15:072.106 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502764218112, Pid 77462]: SyncAllMaps for XmppPeerInfoData:0 period 0 cycle 0 without seqno , total = 0 2025-01-20 Mon 01:50:15:072.138 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502764218112, Pid 77462]: Established 2025-01-20 Mon 01:50:15:143.983 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502953732224, Pid 77462]: TCP [SYS_ERR]: TcpSessionMessageLog: Session 10.0.0.22:40548::10.0.0.49:8086(-1) < Read failed due to error system 104 : Connection reset by peer src/contrail-common/io/tcp_session.cc 507 2025-01-20 Mon 01:50:15:144.019 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502953732224, Pid 77462]: TCP [SYS_ERR]: TcpSessionMessageLog: Session 10.0.0.22:40548::10.0.0.49:8086(-1) > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 312 2025-01-20 Mon 01:50:15:144.041 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502953732224, Pid 77462]: 10.0.0.22:40548::10.0.0.49:8086(-1) OnSessionEvent TCP Connection Closed 2025-01-20 Mon 01:50:15:144.050 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502953732224, Pid 77462]: 10.0.0.22:40548::10.0.0.49:8086(-1):EvTcpClose 2025-01-20 Mon 01:50:15:147.673 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502764218112, Pid 77462]: Processing scm::EvTcpClose in state Established 2025-01-20 Mon 01:50:15:147.713 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502764218112, Pid 77462]: Established : EvTcpClose 2025-01-20 Mon 01:50:15:148.122 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502764218112, Pid 77462]: Connect : Start Connect timer 10.0.0.22:8086 2025-01-20 Mon 01:50:15:148.293 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502764218112, Pid 77462]: Processing scm::EvSandeshSend in state Connect 2025-01-20 Mon 01:50:15:148.660 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502764218112, Pid 77462]: SANDESH: Send FAILED: 1737337815147880 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-2650-2:Control:contrail-dns:0 client_info= [ status = Established successful_connections = 2 pid = 77462 http_port = 8092 start_time = 1737337641318269 collector_name = cn-jenkins-deploy-platform-ansible-os-2650-3. collector_ip = 10.0.0.22:8086 collector_list= [ [ (*_iter40) = 10.0.0.42:8086, (*_iter40) = 10.0.0.49:8086, (*_iter40) = 10.0.0.22:8086, ] ] rx_socket_stats= [ bytes = 2034 calls = 4 average_bytes = 508 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 294780 calls = 24 average_bytes = 12282 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ConfigClientGetRowError [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 6 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigClientMgrDebug [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 648 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigurationMalformedReferenceWarningLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 17 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = DnsOperational [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 14 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapLinkOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 469 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapNodeOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 271 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 6 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 12 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlClientToServer [ messages_sent = 2 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlServerToClient [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 12 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 9 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 12 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveDnsInfo [ messages_sent = 2 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionCreate [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 6 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppCreateConnection [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 1445, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 21, _iter59->first = sent _iter59->second = 28, ] ] ] 2025-01-20 Mon 01:50:15:148.691 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502764218112, Pid 77462]: Wrong state: Connect for event: EvSandeshSend message: SandeshModuleClientTrace 2025-01-20 Mon 01:50:15:148.704 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502764218112, Pid 77462]: Processing scm::EvSandeshSend in state Connect 2025-01-20 Mon 01:50:15:148.728 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502764218112, Pid 77462]: SANDESH: Send FAILED: 1737337815147929 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-2650-2 process_status= [ [ [ module_id = contrail-dns instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.49:8086, ] ] status = Initializing description = Established : EvTcpClose -> Connect ], [ type = Database name = Cassandra server_addrs= [ [ (*_iter6) = 10.0.0.42:9041, (*_iter6) = 10.0.0.22:9041, (*_iter6) = 10.0.0.49:9041, ] ] status = Up description = Established Cassandra connection ], [ type = Database name = RabbitMQ server_addrs= [ [ (*_iter6) = 10.0.0.42:5673, (*_iter6) = 10.0.0.22:5673, (*_iter6) = 10.0.0.49:5673, ] ] status = Up description = RabbitMQ connection established ], ] ] flag_infos= [ [ ] ] description = Collector connection down ], ] ] ] 2025-01-20 Mon 01:50:15:148.737 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502764218112, Pid 77462]: Wrong state: Connect for event: EvSandeshSend message: NodeStatusUVE 2025-01-20 Mon 01:50:15:148.746 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502764218112, Pid 77462]: Processing scm::EvTcpDeleteSession in state Connect 2025-01-20 Mon 01:50:15:148.778 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502764218112, Pid 77462]: Processing scm::EvSandeshSend in state Connect 2025-01-20 Mon 01:50:15:149.003 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502764218112, Pid 77462]: SANDESH: Send FAILED: 1737337815148264 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-2650-2:Control:contrail-dns:0 client_info= [ status = Connect successful_connections = 2 pid = 77462 http_port = 8092 start_time = 1737337641318269 collector_name = collector_ip = 10.0.0.22:8086 collector_list= [ [ (*_iter40) = 10.0.0.42:8086, (*_iter40) = 10.0.0.49:8086, (*_iter40) = 10.0.0.22:8086, ] ] rx_socket_stats= [ bytes = 2034 calls = 4 average_bytes = 508 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 294780 calls = 24 average_bytes = 12282 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ConfigClientGetRowError [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 6 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigClientMgrDebug [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 648 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigurationMalformedReferenceWarningLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 17 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = DnsOperational [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 14 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapLinkOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 469 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapNodeOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 271 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 6 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 12 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlClientToServer [ messages_sent = 2 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlServerToClient [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 12 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 9 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 12 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveDnsInfo [ messages_sent = 2 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionCreate [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 6 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppCreateConnection [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 1445, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 21, _iter59->first = sent _iter59->second = 28, ] ] ] 2025-01-20 Mon 01:50:15:149.019 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502764218112, Pid 77462]: Wrong state: Connect for event: EvSandeshSend message: SandeshModuleClientTrace 2025-01-20 Mon 01:50:15:158.493 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502953732224, Pid 77462]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.0.0.22:37714::10.0.0.22:8086(-1) < Active session connection complete src/contrail-common/io/tcp_session.cc 280 2025-01-20 Mon 01:50:15:158.521 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502953732224, Pid 77462]: 10.0.0.22:37714::10.0.0.22:8086(-1) OnSessionEvent TCP Connected 2025-01-20 Mon 01:50:15:158.529 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502953732224, Pid 77462]: 10.0.0.22:37714::10.0.0.22:8086(-1):EvTcpConnected 2025-01-20 Mon 01:50:15:160.044 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502772614912, Pid 77462]: Processing scm::EvTcpConnected in state Connect 2025-01-20 Mon 01:50:15:160.066 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502772614912, Pid 77462]: Connect : EvTcpConnected : Cancelling Connect timer 2025-01-20 Mon 01:50:15:160.105 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502772614912, Pid 77462]: ClientInit 2025-01-20 Mon 01:50:15:160.117 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502772614912, Pid 77462]: Sending Ctrl Message for cn-jenkins-deploy-platform-ansible-os-2650-2:contrail-dns:0:Control count 3 2025-01-20 Mon 01:50:15:160.540 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502772614912, Pid 77462]: Processing scm::EvSandeshSend in state ClientInit 2025-01-20 Mon 01:50:15:160.564 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502772614912, Pid 77462]: ClientInit : EvSandeshSend : NodeStatusUVE 2025-01-20 Mon 01:50:15:160.598 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502772614912, Pid 77462]: SANDESH: Send FAILED: 1737337815160096 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-2650-2 process_status= [ [ [ module_id = contrail-dns instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.22:8086, ] ] status = Initializing description = Connect : EvTcpConnected ], [ type = Database name = Cassandra server_addrs= [ [ (*_iter6) = 10.0.0.42:9041, (*_iter6) = 10.0.0.22:9041, (*_iter6) = 10.0.0.49:9041, ] ] status = Up description = Established Cassandra connection ], [ type = Database name = RabbitMQ server_addrs= [ [ (*_iter6) = 10.0.0.42:5673, (*_iter6) = 10.0.0.22:5673, (*_iter6) = 10.0.0.49:5673, ] ] status = Up description = RabbitMQ connection established ], ] ] flag_infos= [ [ ] ] description = Collector connection down ], ] ] ] 2025-01-20 Mon 01:50:15:160.609 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502772614912, Pid 77462]: Received UVE message in wrong state : NodeStatusUVE 2025-01-20 Mon 01:50:15:160.622 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502772614912, Pid 77462]: Processing scm::EvSandeshSend in state ClientInit 2025-01-20 Mon 01:50:15:160.632 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502772614912, Pid 77462]: ClientInit : EvSandeshSend : SandeshCtrlClientToServer 2025-01-20 Mon 01:50:15:160.657 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502772614912, Pid 77462]: Processing scm::EvSandeshSend in state ClientInit 2025-01-20 Mon 01:50:15:160.667 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502772614912, Pid 77462]: ClientInit : EvSandeshSend : SandeshModuleClientTrace 2025-01-20 Mon 01:50:15:160.944 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502772614912, Pid 77462]: SANDESH: Send FAILED: 1737337815160514 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-2650-2:Control:contrail-dns:0 client_info= [ status = ClientInit successful_connections = 3 pid = 77462 http_port = 8092 start_time = 1737337641318269 collector_name = collector_ip = 10.0.0.22:8086 collector_list= [ [ (*_iter40) = 10.0.0.42:8086, (*_iter40) = 10.0.0.49:8086, (*_iter40) = 10.0.0.22:8086, ] ] rx_socket_stats= [ bytes = 2034 calls = 4 average_bytes = 508 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 294780 calls = 24 average_bytes = 12282 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ConfigClientGetRowError [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 6 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigClientMgrDebug [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 648 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigurationMalformedReferenceWarningLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 17 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = DnsOperational [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 14 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapLinkOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 469 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapNodeOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 271 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 6 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 13 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlClientToServer [ messages_sent = 2 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlServerToClient [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 12 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 11 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 13 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveDnsInfo [ messages_sent = 2 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionCreate [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 6 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppCreateConnection [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 1446, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 24, _iter59->first = sent _iter59->second = 28, ] ] ] 2025-01-20 Mon 01:50:15:160.971 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502772614912, Pid 77462]: Received UVE message in wrong state : SandeshModuleClientTrace 2025-01-20 Mon 01:50:15:237.565 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502953732224, Pid 77462]: TCP [SYS_ERR]: TcpSessionMessageLog: Session 10.0.0.22:37714::10.0.0.22:8086(-1) < Read failed due to error system 104 : Connection reset by peer src/contrail-common/io/tcp_session.cc 507 2025-01-20 Mon 01:50:15:237.602 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502953732224, Pid 77462]: TCP [SYS_ERR]: TcpSessionMessageLog: Session 10.0.0.22:37714::10.0.0.22:8086(-1) > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 312 2025-01-20 Mon 01:50:15:237.619 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502953732224, Pid 77462]: 10.0.0.22:37714::10.0.0.22:8086(-1) OnSessionEvent TCP Connection Closed 2025-01-20 Mon 01:50:15:237.627 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502953732224, Pid 77462]: 10.0.0.22:37714::10.0.0.22:8086(-1):EvTcpClose 2025-01-20 Mon 01:50:15:239.030 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502797805312, Pid 77462]: Processing scm::EvTcpClose in state ClientInit 2025-01-20 Mon 01:50:15:239.046 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502797805312, Pid 77462]: ClientInit : EvTcpClose 2025-01-20 Mon 01:50:15:239.221 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502797805312, Pid 77462]: Return to idle with 5000 2025-01-20 Mon 01:50:15:239.238 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502797805312, Pid 77462]: Idle 2025-01-20 Mon 01:50:15:239.350 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502797805312, Pid 77462]: Processing scm::EvSandeshSend in state Idle 2025-01-20 Mon 01:50:15:239.745 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502797805312, Pid 77462]: SANDESH: Send FAILED: 1737337815239142 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-2650-2:Control:contrail-dns:0 client_info= [ status = ClientInit successful_connections = 3 pid = 77462 http_port = 8092 start_time = 1737337641318269 collector_name = collector_ip = 10.0.0.42:8086 collector_list= [ [ (*_iter40) = 10.0.0.42:8086, (*_iter40) = 10.0.0.49:8086, (*_iter40) = 10.0.0.22:8086, ] ] rx_socket_stats= [ bytes = 2034 calls = 4 average_bytes = 508 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 296390 calls = 25 average_bytes = 11855 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ConfigClientGetRowError [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 6 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigClientMgrDebug [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 648 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigurationMalformedReferenceWarningLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 17 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = DnsOperational [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 14 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapLinkOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 469 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapNodeOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 271 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 6 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 14 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlClientToServer [ messages_sent = 3 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlServerToClient [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 12 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 12 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 15 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveDnsInfo [ messages_sent = 2 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionCreate [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 6 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppCreateConnection [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 1448, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 26, _iter59->first = sent _iter59->second = 29, ] ] ] 2025-01-20 Mon 01:50:15:239.802 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502797805312, Pid 77462]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-01-20 Mon 01:50:15:239.832 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502797805312, Pid 77462]: Processing scm::EvSandeshSend in state Idle 2025-01-20 Mon 01:50:15:239.870 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502797805312, Pid 77462]: SANDESH: Send FAILED: 1737337815239186 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-2650-2 process_status= [ [ [ module_id = contrail-dns instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.22:8086, ] ] status = Down description = ClientInit : EvTcpClose ], [ type = Database name = Cassandra server_addrs= [ [ (*_iter6) = 10.0.0.42:9041, (*_iter6) = 10.0.0.22:9041, (*_iter6) = 10.0.0.49:9041, ] ] status = Up description = Established Cassandra connection ], [ type = Database name = RabbitMQ server_addrs= [ [ (*_iter6) = 10.0.0.42:5673, (*_iter6) = 10.0.0.22:5673, (*_iter6) = 10.0.0.49:5673, ] ] status = Up description = RabbitMQ connection established ], ] ] flag_infos= [ [ ] ] description = Collector connection down ], ] ] ] 2025-01-20 Mon 01:50:15:239.881 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502797805312, Pid 77462]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-01-20 Mon 01:50:15:239.894 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502797805312, Pid 77462]: Processing scm::EvTcpDeleteSession in state Idle 2025-01-20 Mon 01:50:15:239.935 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502797805312, Pid 77462]: Processing scm::EvSandeshSend in state Idle 2025-01-20 Mon 01:50:15:240.367 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502797805312, Pid 77462]: SANDESH: Send FAILED: 1737337815239332 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-2650-2:Control:contrail-dns:0 client_info= [ status = Idle successful_connections = 3 pid = 77462 http_port = 8092 start_time = 1737337641318269 collector_name = collector_ip = 10.0.0.42:8086 collector_list= [ [ (*_iter40) = 10.0.0.42:8086, (*_iter40) = 10.0.0.49:8086, (*_iter40) = 10.0.0.22:8086, ] ] rx_socket_stats= [ bytes = 2034 calls = 4 average_bytes = 508 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 296390 calls = 25 average_bytes = 11855 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ConfigClientGetRowError [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 6 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigClientMgrDebug [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 648 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigurationMalformedReferenceWarningLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 17 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = DnsOperational [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 14 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapLinkOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 469 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapNodeOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 271 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 6 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 14 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlClientToServer [ messages_sent = 3 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlServerToClient [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 12 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 12 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 15 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveDnsInfo [ messages_sent = 2 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionCreate [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 6 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppCreateConnection [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 1448, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 26, _iter59->first = sent _iter59->second = 29, ] ] ] 2025-01-20 Mon 01:50:15:240.411 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502797805312, Pid 77462]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-01-20 Mon 01:50:20:239.340 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502760019712, Pid 77462]: Processing scm::EvIdleHoldTimerExpired in state Idle 2025-01-20 Mon 01:50:20:239.537 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502760019712, Pid 77462]: Connect : Start Connect timer 10.0.0.42:8086 2025-01-20 Mon 01:50:20:239.661 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502760019712, Pid 77462]: Processing scm::EvSandeshSend in state Connect 2025-01-20 Mon 01:50:20:239.698 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502760019712, Pid 77462]: SANDESH: Send FAILED: 1737337820239417 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-2650-2 process_status= [ [ [ module_id = contrail-dns instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.42:8086, ] ] status = Initializing description = Idle : EvIdleHoldTimerExpired -> Connect ], [ type = Database name = Cassandra server_addrs= [ [ (*_iter6) = 10.0.0.42:9041, (*_iter6) = 10.0.0.22:9041, (*_iter6) = 10.0.0.49:9041, ] ] status = Up description = Established Cassandra connection ], [ type = Database name = RabbitMQ server_addrs= [ [ (*_iter6) = 10.0.0.42:5673, (*_iter6) = 10.0.0.22:5673, (*_iter6) = 10.0.0.49:5673, ] ] status = Up description = RabbitMQ connection established ], ] ] flag_infos= [ [ ] ] description = Collector connection down ], ] ] ] 2025-01-20 Mon 01:50:20:239.707 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502760019712, Pid 77462]: Wrong state: Connect for event: EvSandeshSend message: NodeStatusUVE 2025-01-20 Mon 01:50:20:239.716 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502760019712, Pid 77462]: Processing scm::EvSandeshSend in state Connect 2025-01-20 Mon 01:50:20:239.984 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502760019712, Pid 77462]: SANDESH: Send FAILED: 1737337820239643 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-2650-2:Control:contrail-dns:0 client_info= [ status = Connect successful_connections = 3 pid = 77462 http_port = 8092 start_time = 1737337641318269 collector_name = collector_ip = 10.0.0.42:8086 collector_list= [ [ (*_iter40) = 10.0.0.42:8086, (*_iter40) = 10.0.0.49:8086, (*_iter40) = 10.0.0.22:8086, ] ] rx_socket_stats= [ bytes = 2034 calls = 4 average_bytes = 508 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 296390 calls = 25 average_bytes = 11855 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ConfigClientGetRowError [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 6 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigClientMgrDebug [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 648 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigurationMalformedReferenceWarningLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 17 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = DnsOperational [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 14 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapLinkOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 469 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapNodeOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 271 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 6 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 15 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlClientToServer [ messages_sent = 3 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlServerToClient [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 12 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 14 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 15 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveDnsInfo [ messages_sent = 2 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionCreate [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 6 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppCreateConnection [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 1448, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 29, _iter59->first = sent _iter59->second = 29, ] ] ] 2025-01-20 Mon 01:50:20:240.007 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502760019712, Pid 77462]: Wrong state: Connect for event: EvSandeshSend message: SandeshModuleClientTrace 2025-01-20 Mon 01:50:20:241.852 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502953732224, Pid 77462]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.0.0.22:49328::10.0.0.42:8086(-1) < Active session connection complete src/contrail-common/io/tcp_session.cc 280 2025-01-20 Mon 01:50:20:241.883 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502953732224, Pid 77462]: 10.0.0.22:49328::10.0.0.42:8086(-1) OnSessionEvent TCP Connected 2025-01-20 Mon 01:50:20:241.891 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502953732224, Pid 77462]: 10.0.0.22:49328::10.0.0.42:8086(-1):EvTcpConnected 2025-01-20 Mon 01:50:20:241.946 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502760019712, Pid 77462]: Processing scm::EvTcpConnected in state Connect 2025-01-20 Mon 01:50:20:241.983 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502760019712, Pid 77462]: Connect : EvTcpConnected : Cancelling Connect timer 2025-01-20 Mon 01:50:20:242.027 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502760019712, Pid 77462]: ClientInit 2025-01-20 Mon 01:50:20:242.047 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502760019712, Pid 77462]: Sending Ctrl Message for cn-jenkins-deploy-platform-ansible-os-2650-2:contrail-dns:0:Control count 4 2025-01-20 Mon 01:50:20:242.169 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502760019712, Pid 77462]: Processing scm::EvSandeshSend in state ClientInit 2025-01-20 Mon 01:50:20:242.183 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502760019712, Pid 77462]: ClientInit : EvSandeshSend : NodeStatusUVE 2025-01-20 Mon 01:50:20:242.214 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502760019712, Pid 77462]: SANDESH: Send FAILED: 1737337820242018 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-2650-2 process_status= [ [ [ module_id = contrail-dns instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.42:8086, ] ] status = Initializing description = Connect : EvTcpConnected ], [ type = Database name = Cassandra server_addrs= [ [ (*_iter6) = 10.0.0.42:9041, (*_iter6) = 10.0.0.22:9041, (*_iter6) = 10.0.0.49:9041, ] ] status = Up description = Established Cassandra connection ], [ type = Database name = RabbitMQ server_addrs= [ [ (*_iter6) = 10.0.0.42:5673, (*_iter6) = 10.0.0.22:5673, (*_iter6) = 10.0.0.49:5673, ] ] status = Up description = RabbitMQ connection established ], ] ] flag_infos= [ [ ] ] description = Collector connection down ], ] ] ] 2025-01-20 Mon 01:50:20:242.234 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502760019712, Pid 77462]: Received UVE message in wrong state : NodeStatusUVE 2025-01-20 Mon 01:50:20:242.246 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502760019712, Pid 77462]: Processing scm::EvSandeshSend in state ClientInit 2025-01-20 Mon 01:50:20:242.255 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502760019712, Pid 77462]: ClientInit : EvSandeshSend : SandeshCtrlClientToServer 2025-01-20 Mon 01:50:20:242.268 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502760019712, Pid 77462]: Processing scm::EvSandeshSend in state ClientInit 2025-01-20 Mon 01:50:20:242.277 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502760019712, Pid 77462]: ClientInit : EvSandeshSend : SandeshModuleClientTrace 2025-01-20 Mon 01:50:20:242.582 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502760019712, Pid 77462]: SANDESH: Send FAILED: 1737337820242153 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-2650-2:Control:contrail-dns:0 client_info= [ status = ClientInit successful_connections = 4 pid = 77462 http_port = 8092 start_time = 1737337641318269 collector_name = collector_ip = 10.0.0.42:8086 collector_list= [ [ (*_iter40) = 10.0.0.42:8086, (*_iter40) = 10.0.0.49:8086, (*_iter40) = 10.0.0.22:8086, ] ] rx_socket_stats= [ bytes = 2034 calls = 4 average_bytes = 508 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 296390 calls = 25 average_bytes = 11855 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ConfigClientGetRowError [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 6 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigClientMgrDebug [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 648 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigurationMalformedReferenceWarningLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 17 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = DnsOperational [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 14 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapLinkOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 469 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapNodeOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 271 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 6 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 16 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlClientToServer [ messages_sent = 3 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlServerToClient [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 12 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 15 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 16 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveDnsInfo [ messages_sent = 2 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionCreate [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 6 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppCreateConnection [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 1449, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 31, _iter59->first = sent _iter59->second = 29, ] ] ] 2025-01-20 Mon 01:50:20:242.606 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502760019712, Pid 77462]: Received UVE message in wrong state : SandeshModuleClientTrace 2025-01-20 Mon 01:50:22:242.135 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502768416512, Pid 77462]: 10.0.0.42:8086 EvConnectTimerExpired in state ClientInit 2025-01-20 Mon 01:50:22:242.195 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502768416512, Pid 77462]: Processing scm::EvConnectTimerExpired in state ClientInit 2025-01-20 Mon 01:50:22:242.214 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502768416512, Pid 77462]: ClientInit : EvConnectTimerExpired 2025-01-20 Mon 01:50:22:242.431 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502768416512, Pid 77462]: Return to idle with 5000 2025-01-20 Mon 01:50:22:242.443 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502768416512, Pid 77462]: Idle 2025-01-20 Mon 01:50:22:242.536 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502768416512, Pid 77462]: Processing scm::EvSandeshSend in state Idle 2025-01-20 Mon 01:50:22:242.809 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502768416512, Pid 77462]: SANDESH: Send FAILED: 1737337822242309 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-2650-2:Control:contrail-dns:0 client_info= [ status = ClientInit successful_connections = 4 pid = 77462 http_port = 8092 start_time = 1737337641318269 collector_name = collector_ip = 10.0.0.49:8086 collector_list= [ [ (*_iter40) = 10.0.0.42:8086, (*_iter40) = 10.0.0.49:8086, (*_iter40) = 10.0.0.22:8086, ] ] rx_socket_stats= [ bytes = 2034 calls = 4 average_bytes = 508 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 298000 calls = 26 average_bytes = 11461 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ConfigClientGetRowError [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 6 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigClientMgrDebug [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 648 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigurationMalformedReferenceWarningLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 17 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = DnsOperational [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 14 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapLinkOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 469 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapNodeOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 271 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 6 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 17 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlClientToServer [ messages_sent = 4 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlServerToClient [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 12 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 16 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 16 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveDnsInfo [ messages_sent = 2 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionCreate [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 6 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppCreateConnection [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 1449, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 33, _iter59->first = sent _iter59->second = 30, ] ] ] 2025-01-20 Mon 01:50:22:242.837 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502768416512, Pid 77462]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-01-20 Mon 01:50:22:242.848 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502768416512, Pid 77462]: Processing scm::EvSandeshSend in state Idle 2025-01-20 Mon 01:50:22:242.871 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502768416512, Pid 77462]: SANDESH: Send FAILED: 1737337822242346 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-2650-2 process_status= [ [ [ module_id = contrail-dns instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.42:8086, ] ] status = Down description = ClientInit : EvConnectTimerExpired ], [ type = Database name = Cassandra server_addrs= [ [ (*_iter6) = 10.0.0.42:9041, (*_iter6) = 10.0.0.22:9041, (*_iter6) = 10.0.0.49:9041, ] ] status = Up description = Established Cassandra connection ], [ type = Database name = RabbitMQ server_addrs= [ [ (*_iter6) = 10.0.0.42:5673, (*_iter6) = 10.0.0.22:5673, (*_iter6) = 10.0.0.49:5673, ] ] status = Up description = RabbitMQ connection established ], ] ] flag_infos= [ [ ] ] description = Collector connection down ], ] ] ] 2025-01-20 Mon 01:50:22:242.877 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502768416512, Pid 77462]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-01-20 Mon 01:50:22:242.885 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502768416512, Pid 77462]: Processing scm::EvTcpDeleteSession in state Idle 2025-01-20 Mon 01:50:22:242.912 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502768416512, Pid 77462]: Processing scm::EvSandeshSend in state Idle 2025-01-20 Mon 01:50:22:243.144 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502768416512, Pid 77462]: SANDESH: Send FAILED: 1737337822242522 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-2650-2:Control:contrail-dns:0 client_info= [ status = Idle successful_connections = 4 pid = 77462 http_port = 8092 start_time = 1737337641318269 collector_name = collector_ip = 10.0.0.49:8086 collector_list= [ [ (*_iter40) = 10.0.0.42:8086, (*_iter40) = 10.0.0.49:8086, (*_iter40) = 10.0.0.22:8086, ] ] rx_socket_stats= [ bytes = 2034 calls = 4 average_bytes = 508 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 298000 calls = 26 average_bytes = 11461 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ConfigClientGetRowError [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 6 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigClientMgrDebug [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 648 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigurationMalformedReferenceWarningLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 17 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = DnsOperational [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 14 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapLinkOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 469 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapNodeOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 271 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 6 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 17 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlClientToServer [ messages_sent = 4 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlServerToClient [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 12 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 16 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 16 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveDnsInfo [ messages_sent = 2 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionCreate [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 6 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppCreateConnection [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 1449, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 33, _iter59->first = sent _iter59->second = 30, ] ] ] 2025-01-20 Mon 01:50:22:243.162 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502768416512, Pid 77462]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-01-20 Mon 01:50:24:540.678 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502768416512, Pid 77462]: Processing scm::EvSandeshSend in state Idle 2025-01-20 Mon 01:50:24:540.829 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502768416512, Pid 77462]: SANDESH: Send FAILED: 1737337824540539 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-2650-2:Control:contrail-dns:0 sm_queue_count = 0 sm_stats= [ ev_stats= [ [ [ event = scm::EvCollectorUpdate enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvConnectTimerExpired enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvIdleHoldTimerExpired enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvSandeshMessageRecv enqueues = 2 dequeues = 2 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvSandeshSend enqueues = 66 dequeues = 66 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvStart enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpClose enqueues = 3 dequeues = 3 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpConnected enqueues = 4 dequeues = 4 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpDeleteSession enqueues = 4 dequeues = 4 enqueue_fails = 0 dequeue_fails = 0 ], [ event = enqueues = 83 dequeues = 83 enqueue_fails = 0 dequeue_fails = 0 ], ] ] state = Idle state_since = 1737337822242442 last_state = Idle last_event = scm::EvSandeshSend last_event_at = 1737337822242911 ] max_sm_queue_count = 7 ] 2025-01-20 Mon 01:50:24:540.854 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502768416512, Pid 77462]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-01-20 Mon 01:50:24:540.863 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502768416512, Pid 77462]: Processing scm::EvSandeshSend in state Idle 2025-01-20 Mon 01:50:24:541.120 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502768416512, Pid 77462]: SANDESH: Send FAILED: 1737337824540652 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-2650-2:Control:contrail-dns:0 client_info= [ status = Idle successful_connections = 4 pid = 77462 http_port = 8092 start_time = 1737337641318269 collector_name = collector_ip = 10.0.0.49:8086 collector_list= [ [ (*_iter40) = 10.0.0.42:8086, (*_iter40) = 10.0.0.49:8086, (*_iter40) = 10.0.0.22:8086, ] ] rx_socket_stats= [ bytes = 2034 calls = 4 average_bytes = 508 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 298000 calls = 26 average_bytes = 11461 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ConfigClientGetRowError [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 6 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigClientMgrDebug [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 648 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigurationMalformedReferenceWarningLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 17 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = DnsOperational [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 14 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapLinkOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 469 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapNodeOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 271 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 6 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 18 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlClientToServer [ messages_sent = 4 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlServerToClient [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 12 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 18 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 16 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveDnsInfo [ messages_sent = 2 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionCreate [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 6 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppCreateConnection [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 1449, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 36, _iter59->first = sent _iter59->second = 30, ] ] ] 2025-01-20 Mon 01:50:24:541.147 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502768416512, Pid 77462]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-01-20 Mon 01:50:27:242.593 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502789408512, Pid 77462]: Processing scm::EvIdleHoldTimerExpired in state Idle 2025-01-20 Mon 01:50:27:242.881 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502789408512, Pid 77462]: Connect : Start Connect timer 10.0.0.49:8086 2025-01-20 Mon 01:50:27:243.019 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502789408512, Pid 77462]: Processing scm::EvSandeshSend in state Connect 2025-01-20 Mon 01:50:27:243.059 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502789408512, Pid 77462]: SANDESH: Send FAILED: 1737337827242687 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-2650-2 process_status= [ [ [ module_id = contrail-dns instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.49:8086, ] ] status = Initializing description = Idle : EvIdleHoldTimerExpired -> Connect ], [ type = Database name = Cassandra server_addrs= [ [ (*_iter6) = 10.0.0.42:9041, (*_iter6) = 10.0.0.22:9041, (*_iter6) = 10.0.0.49:9041, ] ] status = Up description = Established Cassandra connection ], [ type = Database name = RabbitMQ server_addrs= [ [ (*_iter6) = 10.0.0.42:5673, (*_iter6) = 10.0.0.22:5673, (*_iter6) = 10.0.0.49:5673, ] ] status = Up description = RabbitMQ connection established ], ] ] flag_infos= [ [ ] ] description = Collector connection down ], ] ] ] 2025-01-20 Mon 01:50:27:243.071 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502789408512, Pid 77462]: Wrong state: Connect for event: EvSandeshSend message: NodeStatusUVE 2025-01-20 Mon 01:50:27:243.082 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502789408512, Pid 77462]: Processing scm::EvSandeshSend in state Connect 2025-01-20 Mon 01:50:27:243.373 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502789408512, Pid 77462]: SANDESH: Send FAILED: 1737337827242998 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-2650-2:Control:contrail-dns:0 client_info= [ status = Connect successful_connections = 4 pid = 77462 http_port = 8092 start_time = 1737337641318269 collector_name = collector_ip = 10.0.0.49:8086 collector_list= [ [ (*_iter40) = 10.0.0.42:8086, (*_iter40) = 10.0.0.49:8086, (*_iter40) = 10.0.0.22:8086, ] ] rx_socket_stats= [ bytes = 2034 calls = 4 average_bytes = 508 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 298000 calls = 26 average_bytes = 11461 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ConfigClientGetRowError [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 6 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigClientMgrDebug [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 648 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigurationMalformedReferenceWarningLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 17 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = DnsOperational [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 14 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapLinkOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 469 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapNodeOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 271 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 6 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 18 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlClientToServer [ messages_sent = 4 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlServerToClient [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 12 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 20 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 16 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveDnsInfo [ messages_sent = 2 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionCreate [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 6 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppCreateConnection [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 1449, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 38, _iter59->first = sent _iter59->second = 30, ] ] ] 2025-01-20 Mon 01:50:27:243.415 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502789408512, Pid 77462]: Wrong state: Connect for event: EvSandeshSend message: SandeshModuleClientTrace 2025-01-20 Mon 01:50:27:245.355 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502953732224, Pid 77462]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.0.0.22:43402::10.0.0.49:8086(-1) < Active session connection complete src/contrail-common/io/tcp_session.cc 280 2025-01-20 Mon 01:50:27:245.388 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502953732224, Pid 77462]: 10.0.0.22:43402::10.0.0.49:8086(-1) OnSessionEvent TCP Connected 2025-01-20 Mon 01:50:27:245.407 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502953732224, Pid 77462]: 10.0.0.22:43402::10.0.0.49:8086(-1):EvTcpConnected 2025-01-20 Mon 01:50:27:245.478 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502776813312, Pid 77462]: Processing scm::EvTcpConnected in state Connect 2025-01-20 Mon 01:50:27:245.507 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502776813312, Pid 77462]: Connect : EvTcpConnected : Cancelling Connect timer 2025-01-20 Mon 01:50:27:245.556 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502776813312, Pid 77462]: ClientInit 2025-01-20 Mon 01:50:27:245.574 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502776813312, Pid 77462]: Sending Ctrl Message for cn-jenkins-deploy-platform-ansible-os-2650-2:contrail-dns:0:Control count 5 2025-01-20 Mon 01:50:27:245.687 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502776813312, Pid 77462]: Processing scm::EvSandeshSend in state ClientInit 2025-01-20 Mon 01:50:27:245.696 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502776813312, Pid 77462]: ClientInit : EvSandeshSend : NodeStatusUVE 2025-01-20 Mon 01:50:27:245.719 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502776813312, Pid 77462]: SANDESH: Send FAILED: 1737337827245548 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-2650-2 process_status= [ [ [ module_id = contrail-dns instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.49:8086, ] ] status = Initializing description = Connect : EvTcpConnected ], [ type = Database name = Cassandra server_addrs= [ [ (*_iter6) = 10.0.0.42:9041, (*_iter6) = 10.0.0.22:9041, (*_iter6) = 10.0.0.49:9041, ] ] status = Up description = Established Cassandra connection ], [ type = Database name = RabbitMQ server_addrs= [ [ (*_iter6) = 10.0.0.42:5673, (*_iter6) = 10.0.0.22:5673, (*_iter6) = 10.0.0.49:5673, ] ] status = Up description = RabbitMQ connection established ], ] ] flag_infos= [ [ ] ] description = Collector connection down ], ] ] ] 2025-01-20 Mon 01:50:27:245.726 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502776813312, Pid 77462]: Received UVE message in wrong state : NodeStatusUVE 2025-01-20 Mon 01:50:27:245.735 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502776813312, Pid 77462]: Processing scm::EvSandeshSend in state ClientInit 2025-01-20 Mon 01:50:27:245.742 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502776813312, Pid 77462]: ClientInit : EvSandeshSend : SandeshCtrlClientToServer 2025-01-20 Mon 01:50:27:245.757 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502776813312, Pid 77462]: Processing scm::EvSandeshSend in state ClientInit 2025-01-20 Mon 01:50:27:245.765 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502776813312, Pid 77462]: ClientInit : EvSandeshSend : SandeshModuleClientTrace 2025-01-20 Mon 01:50:27:246.064 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502776813312, Pid 77462]: SANDESH: Send FAILED: 1737337827245671 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-2650-2:Control:contrail-dns:0 client_info= [ status = ClientInit successful_connections = 5 pid = 77462 http_port = 8092 start_time = 1737337641318269 collector_name = collector_ip = 10.0.0.49:8086 collector_list= [ [ (*_iter40) = 10.0.0.42:8086, (*_iter40) = 10.0.0.49:8086, (*_iter40) = 10.0.0.22:8086, ] ] rx_socket_stats= [ bytes = 2034 calls = 4 average_bytes = 508 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 298000 calls = 26 average_bytes = 11461 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ConfigClientGetRowError [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 6 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigClientMgrDebug [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 648 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigurationMalformedReferenceWarningLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 17 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = DnsOperational [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 14 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapLinkOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 469 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapNodeOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 271 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 6 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 19 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlClientToServer [ messages_sent = 4 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlServerToClient [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 12 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 21 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 17 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveDnsInfo [ messages_sent = 2 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionCreate [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 6 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppCreateConnection [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 1450, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 40, _iter59->first = sent _iter59->second = 30, ] ] ] 2025-01-20 Mon 01:50:27:246.088 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502776813312, Pid 77462]: Received UVE message in wrong state : SandeshModuleClientTrace 2025-01-20 Mon 01:50:28:229.825 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502953732224, Pid 77462]: TCP [SYS_ERR]: TcpSessionMessageLog: Session 10.0.0.22:43402::10.0.0.49:8086(-1) < Read failed due to error system 104 : Connection reset by peer src/contrail-common/io/tcp_session.cc 507 2025-01-20 Mon 01:50:28:237.278 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502953732224, Pid 77462]: TCP [SYS_ERR]: TcpSessionMessageLog: Session 10.0.0.22:43402::10.0.0.49:8086(-1) > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 312 2025-01-20 Mon 01:50:28:237.331 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502953732224, Pid 77462]: 10.0.0.22:43402::10.0.0.49:8086(-1) OnSessionEvent TCP Connection Closed 2025-01-20 Mon 01:50:28:237.343 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502953732224, Pid 77462]: 10.0.0.22:43402::10.0.0.49:8086(-1):EvTcpClose 2025-01-20 Mon 01:50:28:237.446 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502789408512, Pid 77462]: Processing scm::EvTcpClose in state ClientInit 2025-01-20 Mon 01:50:28:237.487 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502789408512, Pid 77462]: ClientInit : EvTcpClose 2025-01-20 Mon 01:50:28:237.690 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502789408512, Pid 77462]: Return to idle with 5000 2025-01-20 Mon 01:50:28:237.703 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502789408512, Pid 77462]: Idle 2025-01-20 Mon 01:50:28:237.835 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502789408512, Pid 77462]: Processing scm::EvSandeshSend in state Idle 2025-01-20 Mon 01:50:28:238.269 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502789408512, Pid 77462]: SANDESH: Send FAILED: 1737337828237629 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-2650-2:Control:contrail-dns:0 client_info= [ status = ClientInit successful_connections = 5 pid = 77462 http_port = 8092 start_time = 1737337641318269 collector_name = collector_ip = 10.0.0.22:8086 collector_list= [ [ (*_iter40) = 10.0.0.42:8086, (*_iter40) = 10.0.0.49:8086, (*_iter40) = 10.0.0.22:8086, ] ] rx_socket_stats= [ bytes = 2034 calls = 4 average_bytes = 508 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 299610 calls = 27 average_bytes = 11096 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ConfigClientGetRowError [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 6 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigClientMgrDebug [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 648 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigurationMalformedReferenceWarningLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 17 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = DnsOperational [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 14 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapLinkOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 469 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapNodeOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 271 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 6 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 20 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlClientToServer [ messages_sent = 5 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlServerToClient [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 12 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 22 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 19 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveDnsInfo [ messages_sent = 2 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionCreate [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 6 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppCreateConnection [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 1452, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 42, _iter59->first = sent _iter59->second = 31, ] ] ] 2025-01-20 Mon 01:50:28:238.296 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502789408512, Pid 77462]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-01-20 Mon 01:50:28:238.315 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502789408512, Pid 77462]: Processing scm::EvSandeshSend in state Idle 2025-01-20 Mon 01:50:28:238.348 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502789408512, Pid 77462]: SANDESH: Send FAILED: 1737337828237672 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-2650-2 process_status= [ [ [ module_id = contrail-dns instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.49:8086, ] ] status = Down description = ClientInit : EvTcpClose ], [ type = Database name = Cassandra server_addrs= [ [ (*_iter6) = 10.0.0.42:9041, (*_iter6) = 10.0.0.22:9041, (*_iter6) = 10.0.0.49:9041, ] ] status = Up description = Established Cassandra connection ], [ type = Database name = RabbitMQ server_addrs= [ [ (*_iter6) = 10.0.0.42:5673, (*_iter6) = 10.0.0.22:5673, (*_iter6) = 10.0.0.49:5673, ] ] status = Up description = RabbitMQ connection established ], ] ] flag_infos= [ [ ] ] description = Collector connection down ], ] ] ] 2025-01-20 Mon 01:50:28:238.357 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502789408512, Pid 77462]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-01-20 Mon 01:50:28:238.369 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502789408512, Pid 77462]: Processing scm::EvTcpDeleteSession in state Idle 2025-01-20 Mon 01:50:28:238.410 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502789408512, Pid 77462]: Processing scm::EvSandeshSend in state Idle 2025-01-20 Mon 01:50:28:238.815 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502789408512, Pid 77462]: SANDESH: Send FAILED: 1737337828237817 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-2650-2:Control:contrail-dns:0 client_info= [ status = Idle successful_connections = 5 pid = 77462 http_port = 8092 start_time = 1737337641318269 collector_name = collector_ip = 10.0.0.22:8086 collector_list= [ [ (*_iter40) = 10.0.0.42:8086, (*_iter40) = 10.0.0.49:8086, (*_iter40) = 10.0.0.22:8086, ] ] rx_socket_stats= [ bytes = 2034 calls = 4 average_bytes = 508 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 299610 calls = 27 average_bytes = 11096 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ConfigClientGetRowError [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 6 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigClientMgrDebug [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 648 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigurationMalformedReferenceWarningLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 17 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = DnsOperational [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 14 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapLinkOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 469 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapNodeOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 271 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 6 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 20 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlClientToServer [ messages_sent = 5 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlServerToClient [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 12 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 22 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 19 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveDnsInfo [ messages_sent = 2 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionCreate [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 6 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppCreateConnection [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 1452, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 42, _iter59->first = sent _iter59->second = 31, ] ] ] 2025-01-20 Mon 01:50:28:238.841 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502789408512, Pid 77462]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-01-20 Mon 01:50:33:237.851 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502781011712, Pid 77462]: Processing scm::EvIdleHoldTimerExpired in state Idle 2025-01-20 Mon 01:50:33:238.088 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502781011712, Pid 77462]: Connect : Start Connect timer 10.0.0.22:8086 2025-01-20 Mon 01:50:33:238.216 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502781011712, Pid 77462]: Processing scm::EvSandeshSend in state Connect 2025-01-20 Mon 01:50:33:238.246 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502781011712, Pid 77462]: SANDESH: Send FAILED: 1737337833237931 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-2650-2 process_status= [ [ [ module_id = contrail-dns instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.22:8086, ] ] status = Initializing description = Idle : EvIdleHoldTimerExpired -> Connect ], [ type = Database name = Cassandra server_addrs= [ [ (*_iter6) = 10.0.0.42:9041, (*_iter6) = 10.0.0.22:9041, (*_iter6) = 10.0.0.49:9041, ] ] status = Up description = Established Cassandra connection ], [ type = Database name = RabbitMQ server_addrs= [ [ (*_iter6) = 10.0.0.42:5673, (*_iter6) = 10.0.0.22:5673, (*_iter6) = 10.0.0.49:5673, ] ] status = Up description = RabbitMQ connection established ], ] ] flag_infos= [ [ ] ] description = Collector connection down ], ] ] ] 2025-01-20 Mon 01:50:33:238.256 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502781011712, Pid 77462]: Wrong state: Connect for event: EvSandeshSend message: NodeStatusUVE 2025-01-20 Mon 01:50:33:238.264 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502781011712, Pid 77462]: Processing scm::EvSandeshSend in state Connect 2025-01-20 Mon 01:50:33:238.525 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502781011712, Pid 77462]: SANDESH: Send FAILED: 1737337833238198 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-2650-2:Control:contrail-dns:0 client_info= [ status = Connect successful_connections = 5 pid = 77462 http_port = 8092 start_time = 1737337641318269 collector_name = collector_ip = 10.0.0.22:8086 collector_list= [ [ (*_iter40) = 10.0.0.42:8086, (*_iter40) = 10.0.0.49:8086, (*_iter40) = 10.0.0.22:8086, ] ] rx_socket_stats= [ bytes = 2034 calls = 4 average_bytes = 508 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 299610 calls = 27 average_bytes = 11096 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ConfigClientGetRowError [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 6 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigClientMgrDebug [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 648 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigurationMalformedReferenceWarningLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 17 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = DnsOperational [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 14 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapLinkOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 469 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapNodeOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 271 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 6 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 21 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlClientToServer [ messages_sent = 5 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlServerToClient [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 12 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 24 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 19 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveDnsInfo [ messages_sent = 2 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionCreate [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 6 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppCreateConnection [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 1452, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 45, _iter59->first = sent _iter59->second = 31, ] ] ] 2025-01-20 Mon 01:50:33:238.548 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502781011712, Pid 77462]: Wrong state: Connect for event: EvSandeshSend message: SandeshModuleClientTrace 2025-01-20 Mon 01:50:33:238.599 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502953732224, Pid 77462]: -(-1) OnSessionEvent TCP Connect Failed 2025-01-20 Mon 01:50:33:238.607 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502953732224, Pid 77462]: -(-1):EvTcpConnectFail 2025-01-20 Mon 01:50:33:238.684 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502755821312, Pid 77462]: Processing scm::EvTcpConnectFail in state Connect 2025-01-20 Mon 01:50:33:238.705 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502755821312, Pid 77462]: Connect : EvTcpConnectFail 2025-01-20 Mon 01:50:33:238.845 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502755821312, Pid 77462]: TCP [SYS_ERR]: TcpSessionMessageLog: Session -(-1) > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 312 2025-01-20 Mon 01:50:33:238.878 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502755821312, Pid 77462]: Idle 2025-01-20 Mon 01:50:33:238.950 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502755821312, Pid 77462]: Processing scm::EvSandeshSend in state Idle 2025-01-20 Mon 01:50:33:239.192 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502755821312, Pid 77462]: SANDESH: Send FAILED: 1737337833238797 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-2650-2:Control:contrail-dns:0 client_info= [ status = Connect successful_connections = 5 pid = 77462 http_port = 8092 start_time = 1737337641318269 collector_name = collector_ip = 10.0.0.42:8086 collector_list= [ [ (*_iter40) = 10.0.0.42:8086, (*_iter40) = 10.0.0.49:8086, (*_iter40) = 10.0.0.22:8086, ] ] rx_socket_stats= [ bytes = 2034 calls = 4 average_bytes = 508 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 299610 calls = 27 average_bytes = 11096 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ConfigClientGetRowError [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 6 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigClientMgrDebug [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 648 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigurationMalformedReferenceWarningLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 17 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = DnsOperational [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 14 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapLinkOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 469 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapNodeOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 271 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 6 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 22 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlClientToServer [ messages_sent = 5 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlServerToClient [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 12 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 25 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 19 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveDnsInfo [ messages_sent = 2 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionCreate [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 6 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppCreateConnection [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 1452, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 47, _iter59->first = sent _iter59->second = 31, ] ] ] 2025-01-20 Mon 01:50:33:239.212 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502755821312, Pid 77462]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-01-20 Mon 01:50:33:239.222 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502755821312, Pid 77462]: Processing scm::EvSandeshSend in state Idle 2025-01-20 Mon 01:50:33:239.243 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502755821312, Pid 77462]: SANDESH: Send FAILED: 1737337833238826 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-2650-2 process_status= [ [ [ module_id = contrail-dns instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], [ type = Database name = Cassandra server_addrs= [ [ (*_iter6) = 10.0.0.42:9041, (*_iter6) = 10.0.0.22:9041, (*_iter6) = 10.0.0.49:9041, ] ] status = Up description = Established Cassandra connection ], [ type = Database name = RabbitMQ server_addrs= [ [ (*_iter6) = 10.0.0.42:5673, (*_iter6) = 10.0.0.22:5673, (*_iter6) = 10.0.0.49:5673, ] ] status = Up description = RabbitMQ connection established ], ] ] flag_infos= [ [ ] ] description = Collector connection down ], ] ] ] 2025-01-20 Mon 01:50:33:239.249 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502755821312, Pid 77462]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-01-20 Mon 01:50:33:239.257 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502755821312, Pid 77462]: Processing scm::EvTcpDeleteSession in state Idle 2025-01-20 Mon 01:50:33:239.279 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502755821312, Pid 77462]: Processing scm::EvSandeshSend in state Idle 2025-01-20 Mon 01:50:33:239.513 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502755821312, Pid 77462]: SANDESH: Send FAILED: 1737337833238940 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-2650-2:Control:contrail-dns:0 client_info= [ status = Idle successful_connections = 5 pid = 77462 http_port = 8092 start_time = 1737337641318269 collector_name = collector_ip = 10.0.0.42:8086 collector_list= [ [ (*_iter40) = 10.0.0.42:8086, (*_iter40) = 10.0.0.49:8086, (*_iter40) = 10.0.0.22:8086, ] ] rx_socket_stats= [ bytes = 2034 calls = 4 average_bytes = 508 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 299610 calls = 27 average_bytes = 11096 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ConfigClientGetRowError [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 6 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigClientMgrDebug [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 648 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigurationMalformedReferenceWarningLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 17 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = DnsOperational [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 14 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapLinkOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 469 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapNodeOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 271 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 6 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 22 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlClientToServer [ messages_sent = 5 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlServerToClient [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 12 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 25 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 20 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveDnsInfo [ messages_sent = 2 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionCreate [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 6 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppCreateConnection [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 1453, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 47, _iter59->first = sent _iter59->second = 31, ] ] ] 2025-01-20 Mon 01:50:33:239.596 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502755821312, Pid 77462]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-01-20 Mon 01:50:41:238.996 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502789408512, Pid 77462]: Processing scm::EvIdleHoldTimerExpired in state Idle 2025-01-20 Mon 01:50:41:239.219 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502789408512, Pid 77462]: Connect : Start Connect timer 10.0.0.42:8086 2025-01-20 Mon 01:50:41:239.332 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502789408512, Pid 77462]: Processing scm::EvSandeshSend in state Connect 2025-01-20 Mon 01:50:41:239.361 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502789408512, Pid 77462]: SANDESH: Send FAILED: 1737337841239076 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-2650-2 process_status= [ [ [ module_id = contrail-dns instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.42:8086, ] ] status = Initializing description = Idle : EvIdleHoldTimerExpired -> Connect ], [ type = Database name = Cassandra server_addrs= [ [ (*_iter6) = 10.0.0.42:9041, (*_iter6) = 10.0.0.22:9041, (*_iter6) = 10.0.0.49:9041, ] ] status = Up description = Established Cassandra connection ], [ type = Database name = RabbitMQ server_addrs= [ [ (*_iter6) = 10.0.0.42:5673, (*_iter6) = 10.0.0.22:5673, (*_iter6) = 10.0.0.49:5673, ] ] status = Up description = RabbitMQ connection established ], ] ] flag_infos= [ [ ] ] description = Collector connection down ], ] ] ] 2025-01-20 Mon 01:50:41:239.370 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502789408512, Pid 77462]: Wrong state: Connect for event: EvSandeshSend message: NodeStatusUVE 2025-01-20 Mon 01:50:41:239.378 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502789408512, Pid 77462]: Processing scm::EvSandeshSend in state Connect 2025-01-20 Mon 01:50:41:239.639 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502789408512, Pid 77462]: SANDESH: Send FAILED: 1737337841239318 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-2650-2:Control:contrail-dns:0 client_info= [ status = Connect successful_connections = 5 pid = 77462 http_port = 8092 start_time = 1737337641318269 collector_name = collector_ip = 10.0.0.42:8086 collector_list= [ [ (*_iter40) = 10.0.0.42:8086, (*_iter40) = 10.0.0.49:8086, (*_iter40) = 10.0.0.22:8086, ] ] rx_socket_stats= [ bytes = 2034 calls = 4 average_bytes = 508 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 299610 calls = 27 average_bytes = 11096 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ConfigClientGetRowError [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 6 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigClientMgrDebug [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 648 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigurationMalformedReferenceWarningLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 17 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = DnsOperational [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 14 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapLinkOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 469 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapNodeOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 271 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 6 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 23 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlClientToServer [ messages_sent = 5 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlServerToClient [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 12 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 27 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 20 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveDnsInfo [ messages_sent = 2 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionCreate [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 6 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppCreateConnection [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 1453, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 50, _iter59->first = sent _iter59->second = 31, ] ] ] 2025-01-20 Mon 01:50:41:239.663 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502789408512, Pid 77462]: Wrong state: Connect for event: EvSandeshSend message: SandeshModuleClientTrace 2025-01-20 Mon 01:50:41:239.650 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502953732224, Pid 77462]: -(-1) OnSessionEvent TCP Connect Failed 2025-01-20 Mon 01:50:41:239.707 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502953732224, Pid 77462]: -(-1):EvTcpConnectFail 2025-01-20 Mon 01:50:41:239.748 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502747424512, Pid 77462]: Processing scm::EvTcpConnectFail in state Connect 2025-01-20 Mon 01:50:41:239.759 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502747424512, Pid 77462]: Connect : EvTcpConnectFail 2025-01-20 Mon 01:50:41:239.907 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502747424512, Pid 77462]: TCP [SYS_ERR]: TcpSessionMessageLog: Session -(-1) > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 312 2025-01-20 Mon 01:50:41:239.939 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502747424512, Pid 77462]: Idle 2025-01-20 Mon 01:50:41:240.009 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502747424512, Pid 77462]: Processing scm::EvSandeshSend in state Idle 2025-01-20 Mon 01:50:41:240.250 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502747424512, Pid 77462]: SANDESH: Send FAILED: 1737337841239861 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-2650-2:Control:contrail-dns:0 client_info= [ status = Connect successful_connections = 5 pid = 77462 http_port = 8092 start_time = 1737337641318269 collector_name = collector_ip = 10.0.0.49:8086 collector_list= [ [ (*_iter40) = 10.0.0.42:8086, (*_iter40) = 10.0.0.49:8086, (*_iter40) = 10.0.0.22:8086, ] ] rx_socket_stats= [ bytes = 2034 calls = 4 average_bytes = 508 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 299610 calls = 27 average_bytes = 11096 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ConfigClientGetRowError [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 6 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigClientMgrDebug [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 648 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigurationMalformedReferenceWarningLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 17 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = DnsOperational [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 14 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapLinkOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 469 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapNodeOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 271 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 6 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 24 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlClientToServer [ messages_sent = 5 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlServerToClient [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 12 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 28 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 20 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveDnsInfo [ messages_sent = 2 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionCreate [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 6 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppCreateConnection [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 1453, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 52, _iter59->first = sent _iter59->second = 31, ] ] ] 2025-01-20 Mon 01:50:41:240.271 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502747424512, Pid 77462]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-01-20 Mon 01:50:41:240.281 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502747424512, Pid 77462]: Processing scm::EvSandeshSend in state Idle 2025-01-20 Mon 01:50:41:240.302 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502747424512, Pid 77462]: SANDESH: Send FAILED: 1737337841239888 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-2650-2 process_status= [ [ [ module_id = contrail-dns instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], [ type = Database name = Cassandra server_addrs= [ [ (*_iter6) = 10.0.0.42:9041, (*_iter6) = 10.0.0.22:9041, (*_iter6) = 10.0.0.49:9041, ] ] status = Up description = Established Cassandra connection ], [ type = Database name = RabbitMQ server_addrs= [ [ (*_iter6) = 10.0.0.42:5673, (*_iter6) = 10.0.0.22:5673, (*_iter6) = 10.0.0.49:5673, ] ] status = Up description = RabbitMQ connection established ], ] ] flag_infos= [ [ ] ] description = Collector connection down ], ] ] ] 2025-01-20 Mon 01:50:41:240.308 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502747424512, Pid 77462]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-01-20 Mon 01:50:41:240.316 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502747424512, Pid 77462]: Processing scm::EvTcpDeleteSession in state Idle 2025-01-20 Mon 01:50:41:240.336 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502747424512, Pid 77462]: Processing scm::EvSandeshSend in state Idle 2025-01-20 Mon 01:50:41:240.576 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502747424512, Pid 77462]: SANDESH: Send FAILED: 1737337841239992 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-2650-2:Control:contrail-dns:0 client_info= [ status = Idle successful_connections = 5 pid = 77462 http_port = 8092 start_time = 1737337641318269 collector_name = collector_ip = 10.0.0.49:8086 collector_list= [ [ (*_iter40) = 10.0.0.42:8086, (*_iter40) = 10.0.0.49:8086, (*_iter40) = 10.0.0.22:8086, ] ] rx_socket_stats= [ bytes = 2034 calls = 4 average_bytes = 508 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 299610 calls = 27 average_bytes = 11096 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ConfigClientGetRowError [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 6 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigClientMgrDebug [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 648 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigurationMalformedReferenceWarningLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 17 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = DnsOperational [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 14 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapLinkOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 469 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapNodeOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 271 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 6 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 24 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlClientToServer [ messages_sent = 5 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlServerToClient [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 12 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 28 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 21 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveDnsInfo [ messages_sent = 2 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionCreate [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 6 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppCreateConnection [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 1454, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 52, _iter59->first = sent _iter59->second = 31, ] ] ] 2025-01-20 Mon 01:50:41:240.594 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502747424512, Pid 77462]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-01-20 Mon 01:50:54:541.015 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502793606912, Pid 77462]: Processing scm::EvSandeshSend in state Idle 2025-01-20 Mon 01:50:54:541.186 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502793606912, Pid 77462]: SANDESH: Send FAILED: 1737337854540801 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-2650-2:Control:contrail-dns:0 sm_queue_count = 0 sm_stats= [ ev_stats= [ [ [ event = scm::EvCollectorUpdate enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvConnectTimerExpired enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvIdleHoldTimerExpired enqueues = 4 dequeues = 4 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvSandeshMessageRecv enqueues = 2 dequeues = 2 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvSandeshSend enqueues = 86 dequeues = 86 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvStart enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpClose enqueues = 4 dequeues = 4 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpConnectFail enqueues = 2 dequeues = 2 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpConnected enqueues = 5 dequeues = 5 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpDeleteSession enqueues = 7 dequeues = 7 enqueue_fails = 0 dequeue_fails = 0 ], [ event = enqueues = 113 dequeues = 113 enqueue_fails = 0 dequeue_fails = 0 ], ] ] state = Idle state_since = 1737337841239938 last_state = Idle last_event = scm::EvSandeshSend last_event_at = 1737337841240335 ] max_sm_queue_count = 7 ] 2025-01-20 Mon 01:50:54:541.222 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502793606912, Pid 77462]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-01-20 Mon 01:50:54:541.236 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502793606912, Pid 77462]: Processing scm::EvSandeshSend in state Idle 2025-01-20 Mon 01:50:54:541.725 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502793606912, Pid 77462]: SANDESH: Send FAILED: 1737337854540958 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-2650-2:Control:contrail-dns:0 client_info= [ status = Idle successful_connections = 5 pid = 77462 http_port = 8092 start_time = 1737337641318269 collector_name = collector_ip = 10.0.0.49:8086 collector_list= [ [ (*_iter40) = 10.0.0.42:8086, (*_iter40) = 10.0.0.49:8086, (*_iter40) = 10.0.0.22:8086, ] ] rx_socket_stats= [ bytes = 2034 calls = 4 average_bytes = 508 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 299610 calls = 27 average_bytes = 11096 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ConfigClientGetRowError [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 6 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigClientMgrDebug [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 648 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigurationMalformedReferenceWarningLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 17 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = DnsOperational [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 14 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapLinkOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 469 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapNodeOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 271 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 6 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 25 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlClientToServer [ messages_sent = 5 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlServerToClient [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 12 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 30 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 21 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveDnsInfo [ messages_sent = 2 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionCreate [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 6 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppCreateConnection [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 1454, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 55, _iter59->first = sent _iter59->second = 31, ] ] ] 2025-01-20 Mon 01:50:54:541.753 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502793606912, Pid 77462]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-01-20 Mon 01:50:57:240.100 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502764218112, Pid 77462]: Processing scm::EvIdleHoldTimerExpired in state Idle 2025-01-20 Mon 01:50:57:240.320 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502764218112, Pid 77462]: Connect : Start Connect timer 10.0.0.49:8086 2025-01-20 Mon 01:50:57:240.461 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502764218112, Pid 77462]: Processing scm::EvSandeshSend in state Connect 2025-01-20 Mon 01:50:57:240.490 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502764218112, Pid 77462]: SANDESH: Send FAILED: 1737337857240177 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-2650-2 process_status= [ [ [ module_id = contrail-dns instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.49:8086, ] ] status = Initializing description = Idle : EvIdleHoldTimerExpired -> Connect ], [ type = Database name = Cassandra server_addrs= [ [ (*_iter6) = 10.0.0.42:9041, (*_iter6) = 10.0.0.22:9041, (*_iter6) = 10.0.0.49:9041, ] ] status = Up description = Established Cassandra connection ], [ type = Database name = RabbitMQ server_addrs= [ [ (*_iter6) = 10.0.0.42:5673, (*_iter6) = 10.0.0.22:5673, (*_iter6) = 10.0.0.49:5673, ] ] status = Up description = RabbitMQ connection established ], ] ] flag_infos= [ [ ] ] description = Collector connection down ], ] ] ] 2025-01-20 Mon 01:50:57:240.499 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502764218112, Pid 77462]: Wrong state: Connect for event: EvSandeshSend message: NodeStatusUVE 2025-01-20 Mon 01:50:57:240.520 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502764218112, Pid 77462]: Processing scm::EvSandeshSend in state Connect 2025-01-20 Mon 01:50:57:240.660 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502953732224, Pid 77462]: -(-1) OnSessionEvent TCP Connect Failed 2025-01-20 Mon 01:50:57:240.693 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502953732224, Pid 77462]: -(-1):EvTcpConnectFail 2025-01-20 Mon 01:50:57:240.781 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502764218112, Pid 77462]: SANDESH: Send FAILED: 1737337857240444 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-2650-2:Control:contrail-dns:0 client_info= [ status = Connect successful_connections = 5 pid = 77462 http_port = 8092 start_time = 1737337641318269 collector_name = collector_ip = 10.0.0.49:8086 collector_list= [ [ (*_iter40) = 10.0.0.42:8086, (*_iter40) = 10.0.0.49:8086, (*_iter40) = 10.0.0.22:8086, ] ] rx_socket_stats= [ bytes = 2034 calls = 4 average_bytes = 508 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 299610 calls = 27 average_bytes = 11096 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ConfigClientGetRowError [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 6 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigClientMgrDebug [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 648 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigurationMalformedReferenceWarningLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 17 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = DnsOperational [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 14 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapLinkOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 469 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapNodeOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 271 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 6 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 25 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlClientToServer [ messages_sent = 5 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlServerToClient [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 12 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 32 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 21 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveDnsInfo [ messages_sent = 2 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionCreate [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 6 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppCreateConnection [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 1454, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 57, _iter59->first = sent _iter59->second = 31, ] ] ] 2025-01-20 Mon 01:50:57:240.810 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502764218112, Pid 77462]: Wrong state: Connect for event: EvSandeshSend message: SandeshModuleClientTrace 2025-01-20 Mon 01:50:57:240.828 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502764218112, Pid 77462]: Processing scm::EvTcpConnectFail in state Connect 2025-01-20 Mon 01:50:57:240.836 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502764218112, Pid 77462]: Connect : EvTcpConnectFail 2025-01-20 Mon 01:50:57:240.941 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502764218112, Pid 77462]: TCP [SYS_ERR]: TcpSessionMessageLog: Session -(-1) > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 312 2025-01-20 Mon 01:50:57:240.980 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502764218112, Pid 77462]: Idle 2025-01-20 Mon 01:50:57:241.042 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502764218112, Pid 77462]: Processing scm::EvSandeshSend in state Idle 2025-01-20 Mon 01:50:57:241.266 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502764218112, Pid 77462]: SANDESH: Send FAILED: 1737337857240896 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-2650-2:Control:contrail-dns:0 client_info= [ status = Connect successful_connections = 5 pid = 77462 http_port = 8092 start_time = 1737337641318269 collector_name = collector_ip = 10.0.0.22:8086 collector_list= [ [ (*_iter40) = 10.0.0.42:8086, (*_iter40) = 10.0.0.49:8086, (*_iter40) = 10.0.0.22:8086, ] ] rx_socket_stats= [ bytes = 2034 calls = 4 average_bytes = 508 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 299610 calls = 27 average_bytes = 11096 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ConfigClientGetRowError [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 6 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigClientMgrDebug [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 648 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigurationMalformedReferenceWarningLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 17 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = DnsOperational [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 14 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapLinkOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 469 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapNodeOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 271 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 6 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 26 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlClientToServer [ messages_sent = 5 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlServerToClient [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 12 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 33 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 21 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveDnsInfo [ messages_sent = 2 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionCreate [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 6 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppCreateConnection [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 1454, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 59, _iter59->first = sent _iter59->second = 31, ] ] ] 2025-01-20 Mon 01:50:57:241.289 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502764218112, Pid 77462]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-01-20 Mon 01:50:57:241.302 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502764218112, Pid 77462]: Processing scm::EvSandeshSend in state Idle 2025-01-20 Mon 01:50:57:241.326 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502764218112, Pid 77462]: SANDESH: Send FAILED: 1737337857240920 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-2650-2 process_status= [ [ [ module_id = contrail-dns instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], [ type = Database name = Cassandra server_addrs= [ [ (*_iter6) = 10.0.0.42:9041, (*_iter6) = 10.0.0.22:9041, (*_iter6) = 10.0.0.49:9041, ] ] status = Up description = Established Cassandra connection ], [ type = Database name = RabbitMQ server_addrs= [ [ (*_iter6) = 10.0.0.42:5673, (*_iter6) = 10.0.0.22:5673, (*_iter6) = 10.0.0.49:5673, ] ] status = Up description = RabbitMQ connection established ], ] ] flag_infos= [ [ ] ] description = Collector connection down ], ] ] ] 2025-01-20 Mon 01:50:57:241.332 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502764218112, Pid 77462]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-01-20 Mon 01:50:57:241.341 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502764218112, Pid 77462]: Processing scm::EvTcpDeleteSession in state Idle 2025-01-20 Mon 01:50:57:241.360 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502764218112, Pid 77462]: Processing scm::EvSandeshSend in state Idle 2025-01-20 Mon 01:50:57:241.593 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502764218112, Pid 77462]: SANDESH: Send FAILED: 1737337857241030 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-2650-2:Control:contrail-dns:0 client_info= [ status = Idle successful_connections = 5 pid = 77462 http_port = 8092 start_time = 1737337641318269 collector_name = collector_ip = 10.0.0.22:8086 collector_list= [ [ (*_iter40) = 10.0.0.42:8086, (*_iter40) = 10.0.0.49:8086, (*_iter40) = 10.0.0.22:8086, ] ] rx_socket_stats= [ bytes = 2034 calls = 4 average_bytes = 508 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 299610 calls = 27 average_bytes = 11096 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ConfigClientGetRowError [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 6 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigClientMgrDebug [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 648 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigurationMalformedReferenceWarningLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 17 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = DnsOperational [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 14 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapLinkOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 469 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapNodeOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 271 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 6 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 26 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlClientToServer [ messages_sent = 5 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlServerToClient [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 12 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 33 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 22 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveDnsInfo [ messages_sent = 2 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionCreate [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 6 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppCreateConnection [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 1455, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 59, _iter59->first = sent _iter59->second = 31, ] ] ] 2025-01-20 Mon 01:50:57:241.610 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502764218112, Pid 77462]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-01-20 Mon 01:51:24:541.243 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502797805312, Pid 77462]: Processing scm::EvSandeshSend in state Idle 2025-01-20 Mon 01:51:24:541.434 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502797805312, Pid 77462]: SANDESH: Send FAILED: 1737337884541085 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-2650-2:Control:contrail-dns:0 sm_queue_count = 0 sm_stats= [ ev_stats= [ [ [ event = scm::EvCollectorUpdate enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvConnectTimerExpired enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvIdleHoldTimerExpired enqueues = 5 dequeues = 5 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvSandeshMessageRecv enqueues = 2 dequeues = 2 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvSandeshSend enqueues = 93 dequeues = 93 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvStart enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpClose enqueues = 4 dequeues = 4 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpConnectFail enqueues = 3 dequeues = 3 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpConnected enqueues = 5 dequeues = 5 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpDeleteSession enqueues = 8 dequeues = 8 enqueue_fails = 0 dequeue_fails = 0 ], [ event = enqueues = 123 dequeues = 123 enqueue_fails = 0 dequeue_fails = 0 ], ] ] state = Idle state_since = 1737337857240978 last_state = Idle last_event = scm::EvSandeshSend last_event_at = 1737337857241358 ] max_sm_queue_count = 7 ] 2025-01-20 Mon 01:51:24:541.457 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502797805312, Pid 77462]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-01-20 Mon 01:51:24:541.472 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502797805312, Pid 77462]: Processing scm::EvSandeshSend in state Idle 2025-01-20 Mon 01:51:24:541.820 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502797805312, Pid 77462]: SANDESH: Send FAILED: 1737337884541192 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-2650-2:Control:contrail-dns:0 client_info= [ status = Idle successful_connections = 5 pid = 77462 http_port = 8092 start_time = 1737337641318269 collector_name = collector_ip = 10.0.0.22:8086 collector_list= [ [ (*_iter40) = 10.0.0.42:8086, (*_iter40) = 10.0.0.49:8086, (*_iter40) = 10.0.0.22:8086, ] ] rx_socket_stats= [ bytes = 2034 calls = 4 average_bytes = 508 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 299610 calls = 27 average_bytes = 11096 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ConfigClientGetRowError [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 6 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigClientMgrDebug [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 648 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigurationMalformedReferenceWarningLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 17 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = DnsOperational [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 14 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapLinkOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 469 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapNodeOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 271 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 6 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 27 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlClientToServer [ messages_sent = 5 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlServerToClient [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 12 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 35 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 22 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveDnsInfo [ messages_sent = 2 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionCreate [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 6 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppCreateConnection [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 1455, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 62, _iter59->first = sent _iter59->second = 31, ] ] ] 2025-01-20 Mon 01:51:24:541.854 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502797805312, Pid 77462]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-01-20 Mon 01:51:27:241.075 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502747424512, Pid 77462]: Processing scm::EvIdleHoldTimerExpired in state Idle 2025-01-20 Mon 01:51:27:241.318 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502747424512, Pid 77462]: Connect : Start Connect timer 10.0.0.22:8086 2025-01-20 Mon 01:51:27:241.472 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502747424512, Pid 77462]: Processing scm::EvSandeshSend in state Connect 2025-01-20 Mon 01:51:27:241.516 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502747424512, Pid 77462]: SANDESH: Send FAILED: 1737337887241151 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-2650-2 process_status= [ [ [ module_id = contrail-dns instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.22:8086, ] ] status = Initializing description = Idle : EvIdleHoldTimerExpired -> Connect ], [ type = Database name = Cassandra server_addrs= [ [ (*_iter6) = 10.0.0.42:9041, (*_iter6) = 10.0.0.22:9041, (*_iter6) = 10.0.0.49:9041, ] ] status = Up description = Established Cassandra connection ], [ type = Database name = RabbitMQ server_addrs= [ [ (*_iter6) = 10.0.0.42:5673, (*_iter6) = 10.0.0.22:5673, (*_iter6) = 10.0.0.49:5673, ] ] status = Up description = RabbitMQ connection established ], ] ] flag_infos= [ [ ] ] description = Collector connection down ], ] ] ] 2025-01-20 Mon 01:51:27:241.536 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502747424512, Pid 77462]: Wrong state: Connect for event: EvSandeshSend message: NodeStatusUVE 2025-01-20 Mon 01:51:27:241.548 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502747424512, Pid 77462]: Processing scm::EvSandeshSend in state Connect 2025-01-20 Mon 01:51:27:241.793 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502747424512, Pid 77462]: SANDESH: Send FAILED: 1737337887241453 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-2650-2:Control:contrail-dns:0 client_info= [ status = Connect successful_connections = 5 pid = 77462 http_port = 8092 start_time = 1737337641318269 collector_name = collector_ip = 10.0.0.22:8086 collector_list= [ [ (*_iter40) = 10.0.0.42:8086, (*_iter40) = 10.0.0.49:8086, (*_iter40) = 10.0.0.22:8086, ] ] rx_socket_stats= [ bytes = 2034 calls = 4 average_bytes = 508 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 299610 calls = 27 average_bytes = 11096 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ConfigClientGetRowError [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 6 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigClientMgrDebug [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 648 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigurationMalformedReferenceWarningLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 17 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = DnsOperational [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 14 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapLinkOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 469 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapNodeOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 271 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 6 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 27 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlClientToServer [ messages_sent = 5 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlServerToClient [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 12 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 37 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 22 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveDnsInfo [ messages_sent = 2 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionCreate [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 6 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppCreateConnection [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 1455, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 64, _iter59->first = sent _iter59->second = 31, ] ] ] 2025-01-20 Mon 01:51:27:241.813 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502747424512, Pid 77462]: Wrong state: Connect for event: EvSandeshSend message: SandeshModuleClientTrace 2025-01-20 Mon 01:51:27:241.865 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502953732224, Pid 77462]: -(-1) OnSessionEvent TCP Connect Failed 2025-01-20 Mon 01:51:27:241.878 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502953732224, Pid 77462]: -(-1):EvTcpConnectFail 2025-01-20 Mon 01:51:27:241.911 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502768416512, Pid 77462]: Processing scm::EvTcpConnectFail in state Connect 2025-01-20 Mon 01:51:27:241.941 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502768416512, Pid 77462]: Connect : EvTcpConnectFail 2025-01-20 Mon 01:51:27:242.089 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502768416512, Pid 77462]: TCP [SYS_ERR]: TcpSessionMessageLog: Session -(-1) > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 312 2025-01-20 Mon 01:51:27:242.120 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502768416512, Pid 77462]: Idle 2025-01-20 Mon 01:51:27:242.191 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502768416512, Pid 77462]: Processing scm::EvSandeshSend in state Idle 2025-01-20 Mon 01:51:27:242.469 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502768416512, Pid 77462]: SANDESH: Send FAILED: 1737337887242034 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-2650-2:Control:contrail-dns:0 client_info= [ status = Connect successful_connections = 5 pid = 77462 http_port = 8092 start_time = 1737337641318269 collector_name = collector_ip = 10.0.0.42:8086 collector_list= [ [ (*_iter40) = 10.0.0.42:8086, (*_iter40) = 10.0.0.49:8086, (*_iter40) = 10.0.0.22:8086, ] ] rx_socket_stats= [ bytes = 2034 calls = 4 average_bytes = 508 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 299610 calls = 27 average_bytes = 11096 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ConfigClientGetRowError [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 6 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigClientMgrDebug [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 648 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigurationMalformedReferenceWarningLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 17 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = DnsOperational [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 14 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapLinkOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 469 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapNodeOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 271 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 6 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 28 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlClientToServer [ messages_sent = 5 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlServerToClient [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 12 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 38 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 22 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveDnsInfo [ messages_sent = 2 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionCreate [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 6 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppCreateConnection [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 1455, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 66, _iter59->first = sent _iter59->second = 31, ] ] ] 2025-01-20 Mon 01:51:27:242.495 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502768416512, Pid 77462]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-01-20 Mon 01:51:27:242.505 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502768416512, Pid 77462]: Processing scm::EvSandeshSend in state Idle 2025-01-20 Mon 01:51:27:242.527 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502768416512, Pid 77462]: SANDESH: Send FAILED: 1737337887242065 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-2650-2 process_status= [ [ [ module_id = contrail-dns instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], [ type = Database name = Cassandra server_addrs= [ [ (*_iter6) = 10.0.0.42:9041, (*_iter6) = 10.0.0.22:9041, (*_iter6) = 10.0.0.49:9041, ] ] status = Up description = Established Cassandra connection ], [ type = Database name = RabbitMQ server_addrs= [ [ (*_iter6) = 10.0.0.42:5673, (*_iter6) = 10.0.0.22:5673, (*_iter6) = 10.0.0.49:5673, ] ] status = Up description = RabbitMQ connection established ], ] ] flag_infos= [ [ ] ] description = Collector connection down ], ] ] ] 2025-01-20 Mon 01:51:27:242.534 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502768416512, Pid 77462]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-01-20 Mon 01:51:27:242.542 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502768416512, Pid 77462]: Processing scm::EvTcpDeleteSession in state Idle 2025-01-20 Mon 01:51:27:242.566 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502768416512, Pid 77462]: Processing scm::EvSandeshSend in state Idle 2025-01-20 Mon 01:51:27:242.786 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502768416512, Pid 77462]: SANDESH: Send FAILED: 1737337887242177 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-2650-2:Control:contrail-dns:0 client_info= [ status = Idle successful_connections = 5 pid = 77462 http_port = 8092 start_time = 1737337641318269 collector_name = collector_ip = 10.0.0.42:8086 collector_list= [ [ (*_iter40) = 10.0.0.42:8086, (*_iter40) = 10.0.0.49:8086, (*_iter40) = 10.0.0.22:8086, ] ] rx_socket_stats= [ bytes = 2034 calls = 4 average_bytes = 508 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 299610 calls = 27 average_bytes = 11096 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ConfigClientGetRowError [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 6 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigClientMgrDebug [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 648 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigurationMalformedReferenceWarningLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 17 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = DnsOperational [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 14 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapLinkOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 469 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapNodeOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 271 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 6 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 28 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlClientToServer [ messages_sent = 5 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlServerToClient [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 12 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 38 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 23 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveDnsInfo [ messages_sent = 2 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionCreate [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 6 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppCreateConnection [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 1456, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 66, _iter59->first = sent _iter59->second = 31, ] ] ] 2025-01-20 Mon 01:51:27:242.801 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502768416512, Pid 77462]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-01-20 Mon 01:51:36:368.414 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502755821312, Pid 77462]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 571 2025-01-20 Mon 01:51:36:642.429 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502760019712, Pid 77462]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 571 2025-01-20 Mon 01:51:37:316.637 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502772614912, Pid 77462]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 571 2025-01-20 Mon 01:51:37:419.093 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502772614912, Pid 77462]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 571 2025-01-20 Mon 01:51:37:693.548 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502785210112, Pid 77462]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 571 2025-01-20 Mon 01:51:37:938.588 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502772614912, Pid 77462]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 571 2025-01-20 Mon 01:51:41:297.181 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502781011712, Pid 77462]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 571 2025-01-20 Mon 01:51:41:527.554 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502785210112, Pid 77462]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 571 2025-01-20 Mon 01:51:41:840.558 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502781011712, Pid 77462]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 571 2025-01-20 Mon 01:51:46:688.186 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502755821312, Pid 77462]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 571 2025-01-20 Mon 01:51:52:204.353 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502781011712, Pid 77462]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 571 2025-01-20 Mon 01:51:52:243.878 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502793606912, Pid 77462]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 571 2025-01-20 Mon 01:51:52:278.241 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502781011712, Pid 77462]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 571 2025-01-20 Mon 01:51:54:020.224 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502785210112, Pid 77462]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 571 2025-01-20 Mon 01:51:54:094.793 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502793606912, Pid 77462]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 571 2025-01-20 Mon 01:51:54:159.782 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502772614912, Pid 77462]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 571 2025-01-20 Mon 01:51:54:205.176 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502776813312, Pid 77462]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 571 2025-01-20 Mon 01:51:54:280.845 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502776813312, Pid 77462]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 571 2025-01-20 Mon 01:51:54:353.865 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502776813312, Pid 77462]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 571 2025-01-20 Mon 01:51:54:384.753 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502781011712, Pid 77462]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 571 2025-01-20 Mon 01:51:54:449.605 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502776813312, Pid 77462]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 571 2025-01-20 Mon 01:51:54:525.219 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502781011712, Pid 77462]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 571 2025-01-20 Mon 01:51:54:541.588 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502768416512, Pid 77462]: Processing scm::EvSandeshSend in state Idle 2025-01-20 Mon 01:51:54:541.693 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502768416512, Pid 77462]: SANDESH: Send FAILED: 1737337914541433 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-2650-2:Control:contrail-dns:0 sm_queue_count = 0 sm_stats= [ ev_stats= [ [ [ event = scm::EvCollectorUpdate enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvConnectTimerExpired enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvIdleHoldTimerExpired enqueues = 6 dequeues = 6 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvSandeshMessageRecv enqueues = 2 dequeues = 2 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvSandeshSend enqueues = 100 dequeues = 100 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvStart enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpClose enqueues = 4 dequeues = 4 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpConnectFail enqueues = 4 dequeues = 4 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpConnected enqueues = 5 dequeues = 5 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpDeleteSession enqueues = 9 dequeues = 9 enqueue_fails = 0 dequeue_fails = 0 ], [ event = enqueues = 133 dequeues = 133 enqueue_fails = 0 dequeue_fails = 0 ], ] ] state = Idle state_since = 1737337887242119 last_state = Idle last_event = scm::EvSandeshSend last_event_at = 1737337887242564 ] max_sm_queue_count = 7 ] 2025-01-20 Mon 01:51:54:541.709 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502768416512, Pid 77462]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-01-20 Mon 01:51:54:541.731 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502768416512, Pid 77462]: Processing scm::EvSandeshSend in state Idle 2025-01-20 Mon 01:51:54:542.003 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502768416512, Pid 77462]: SANDESH: Send FAILED: 1737337914541549 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-2650-2:Control:contrail-dns:0 client_info= [ status = Idle successful_connections = 5 pid = 77462 http_port = 8092 start_time = 1737337641318269 collector_name = collector_ip = 10.0.0.42:8086 collector_list= [ [ (*_iter40) = 10.0.0.42:8086, (*_iter40) = 10.0.0.49:8086, (*_iter40) = 10.0.0.22:8086, ] ] rx_socket_stats= [ bytes = 2034 calls = 4 average_bytes = 508 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 299610 calls = 27 average_bytes = 11096 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ConfigClientGetRowError [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 6 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigClientMgrDebug [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 670 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigurationMalformedReferenceWarningLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 17 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = DnsOperational [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 14 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapLinkOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 469 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapNodeOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 271 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 6 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 29 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlClientToServer [ messages_sent = 5 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlServerToClient [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 12 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 40 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 23 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveDnsInfo [ messages_sent = 2 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionCreate [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 6 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppCreateConnection [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 1478, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 69, _iter59->first = sent _iter59->second = 31, ] ] ] 2025-01-20 Mon 01:51:54:542.025 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502768416512, Pid 77462]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-01-20 Mon 01:51:54:557.268 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502768416512, Pid 77462]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 571 2025-01-20 Mon 01:51:54:614.210 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502789408512, Pid 77462]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 571 2025-01-20 Mon 01:51:54:724.418 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502789408512, Pid 77462]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 571 2025-01-20 Mon 01:51:54:752.787 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502776813312, Pid 77462]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 571 2025-01-20 Mon 01:51:54:814.431 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502747424512, Pid 77462]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 571 2025-01-20 Mon 01:51:54:907.820 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502797805312, Pid 77462]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 571 2025-01-20 Mon 01:51:54:989.628 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502797805312, Pid 77462]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 571 2025-01-20 Mon 01:51:55:014.658 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502772614912, Pid 77462]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 571 2025-01-20 Mon 01:51:55:109.269 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502768416512, Pid 77462]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 571 2025-01-20 Mon 01:51:55:181.838 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502747424512, Pid 77462]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 571 2025-01-20 Mon 01:51:55:197.704 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502793606912, Pid 77462]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 571 2025-01-20 Mon 01:51:55:335.743 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502764218112, Pid 77462]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 571 2025-01-20 Mon 01:51:55:382.193 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502768416512, Pid 77462]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 571 2025-01-20 Mon 01:51:55:414.692 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502789408512, Pid 77462]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 571 2025-01-20 Mon 01:51:55:544.470 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502768416512, Pid 77462]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 571 2025-01-20 Mon 01:51:55:573.415 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502755821312, Pid 77462]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 571 2025-01-20 Mon 01:51:55:593.743 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502785210112, Pid 77462]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 571 2025-01-20 Mon 01:51:55:714.680 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502764218112, Pid 77462]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 571 2025-01-20 Mon 01:51:55:800.760 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502797805312, Pid 77462]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 571 2025-01-20 Mon 01:51:55:806.274 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502797805312, Pid 77462]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 571 2025-01-20 Mon 01:51:55:957.540 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502760019712, Pid 77462]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 571 2025-01-20 Mon 01:51:56:040.324 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502747424512, Pid 77462]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 571 2025-01-20 Mon 01:51:56:048.894 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502760019712, Pid 77462]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 571 2025-01-20 Mon 01:51:56:147.775 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502760019712, Pid 77462]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 571 2025-01-20 Mon 01:51:56:222.145 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502768416512, Pid 77462]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 571 2025-01-20 Mon 01:51:56:250.825 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502785210112, Pid 77462]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 571 2025-01-20 Mon 01:51:56:330.810 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502764218112, Pid 77462]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 571 2025-01-20 Mon 01:51:56:398.794 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502781011712, Pid 77462]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 571 2025-01-20 Mon 01:51:56:429.355 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502768416512, Pid 77462]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 571 2025-01-20 Mon 01:51:56:530.767 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502760019712, Pid 77462]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 571 2025-01-20 Mon 01:51:56:599.674 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502760019712, Pid 77462]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 571 2025-01-20 Mon 01:51:56:612.040 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502772614912, Pid 77462]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 571 2025-01-20 Mon 01:51:56:746.836 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502789408512, Pid 77462]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 571 2025-01-20 Mon 01:51:56:813.089 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502789408512, Pid 77462]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 571 2025-01-20 Mon 01:51:56:819.982 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502776813312, Pid 77462]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 571 2025-01-20 Mon 01:51:56:965.265 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502776813312, Pid 77462]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 571 2025-01-20 Mon 01:51:57:004.599 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502768416512, Pid 77462]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 571 2025-01-20 Mon 01:51:57:015.849 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502760019712, Pid 77462]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 571 2025-01-20 Mon 01:51:57:133.713 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502768416512, Pid 77462]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 571 2025-01-20 Mon 01:51:57:166.289 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502764218112, Pid 77462]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 571 2025-01-20 Mon 01:51:57:193.590 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502747424512, Pid 77462]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 571 2025-01-20 Mon 01:51:57:242.216 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502793606912, Pid 77462]: Processing scm::EvIdleHoldTimerExpired in state Idle 2025-01-20 Mon 01:51:57:242.447 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502793606912, Pid 77462]: Connect : Start Connect timer 10.0.0.42:8086 2025-01-20 Mon 01:51:57:242.583 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502793606912, Pid 77462]: Processing scm::EvSandeshSend in state Connect 2025-01-20 Mon 01:51:57:242.614 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502793606912, Pid 77462]: SANDESH: Send FAILED: 1737337917242298 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-2650-2 process_status= [ [ [ module_id = contrail-dns instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.42:8086, ] ] status = Initializing description = Idle : EvIdleHoldTimerExpired -> Connect ], [ type = Database name = Cassandra server_addrs= [ [ (*_iter6) = 10.0.0.42:9041, (*_iter6) = 10.0.0.22:9041, (*_iter6) = 10.0.0.49:9041, ] ] status = Up description = Established Cassandra connection ], [ type = Database name = RabbitMQ server_addrs= [ [ (*_iter6) = 10.0.0.42:5673, (*_iter6) = 10.0.0.22:5673, (*_iter6) = 10.0.0.49:5673, ] ] status = Up description = RabbitMQ connection established ], ] ] flag_infos= [ [ ] ] description = Collector connection down ], ] ] ] 2025-01-20 Mon 01:51:57:242.623 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502793606912, Pid 77462]: Wrong state: Connect for event: EvSandeshSend message: NodeStatusUVE 2025-01-20 Mon 01:51:57:242.631 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502793606912, Pid 77462]: Processing scm::EvSandeshSend in state Connect 2025-01-20 Mon 01:51:57:242.763 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502953732224, Pid 77462]: -(-1) OnSessionEvent TCP Connect Failed 2025-01-20 Mon 01:51:57:242.792 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502953732224, Pid 77462]: -(-1):EvTcpConnectFail 2025-01-20 Mon 01:51:57:242.881 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502793606912, Pid 77462]: SANDESH: Send FAILED: 1737337917242567 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-2650-2:Control:contrail-dns:0 client_info= [ status = Connect successful_connections = 5 pid = 77462 http_port = 8092 start_time = 1737337641318269 collector_name = collector_ip = 10.0.0.42:8086 collector_list= [ [ (*_iter40) = 10.0.0.42:8086, (*_iter40) = 10.0.0.49:8086, (*_iter40) = 10.0.0.22:8086, ] ] rx_socket_stats= [ bytes = 2034 calls = 4 average_bytes = 508 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 299610 calls = 27 average_bytes = 11096 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ConfigClientGetRowError [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 6 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigClientMgrDebug [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 711 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigurationMalformedReferenceWarningLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 17 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = DnsOperational [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 14 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapLinkOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 469 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapNodeOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 271 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 6 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 29 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlClientToServer [ messages_sent = 5 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlServerToClient [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 12 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 42 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 23 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveDnsInfo [ messages_sent = 2 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionCreate [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 6 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppCreateConnection [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 1519, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 71, _iter59->first = sent _iter59->second = 31, ] ] ] 2025-01-20 Mon 01:51:57:242.902 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502793606912, Pid 77462]: Wrong state: Connect for event: EvSandeshSend message: SandeshModuleClientTrace 2025-01-20 Mon 01:51:57:242.912 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502793606912, Pid 77462]: Processing scm::EvTcpConnectFail in state Connect 2025-01-20 Mon 01:51:57:242.919 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502793606912, Pid 77462]: Connect : EvTcpConnectFail 2025-01-20 Mon 01:51:57:243.016 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502793606912, Pid 77462]: TCP [SYS_ERR]: TcpSessionMessageLog: Session -(-1) > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 312 2025-01-20 Mon 01:51:57:243.045 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502793606912, Pid 77462]: Idle 2025-01-20 Mon 01:51:57:243.107 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502793606912, Pid 77462]: Processing scm::EvSandeshSend in state Idle 2025-01-20 Mon 01:51:57:243.333 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502793606912, Pid 77462]: SANDESH: Send FAILED: 1737337917242973 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-2650-2:Control:contrail-dns:0 client_info= [ status = Connect successful_connections = 5 pid = 77462 http_port = 8092 start_time = 1737337641318269 collector_name = collector_ip = 10.0.0.49:8086 collector_list= [ [ (*_iter40) = 10.0.0.42:8086, (*_iter40) = 10.0.0.49:8086, (*_iter40) = 10.0.0.22:8086, ] ] rx_socket_stats= [ bytes = 2034 calls = 4 average_bytes = 508 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 299610 calls = 27 average_bytes = 11096 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ConfigClientGetRowError [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 6 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigClientMgrDebug [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 711 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigurationMalformedReferenceWarningLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 17 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = DnsOperational [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 14 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapLinkOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 469 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapNodeOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 271 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 6 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 30 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlClientToServer [ messages_sent = 5 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlServerToClient [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 12 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 43 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 23 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveDnsInfo [ messages_sent = 2 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionCreate [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 6 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppCreateConnection [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 1519, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 73, _iter59->first = sent _iter59->second = 31, ] ] ] 2025-01-20 Mon 01:51:57:243.352 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502793606912, Pid 77462]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-01-20 Mon 01:51:57:243.361 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502793606912, Pid 77462]: Processing scm::EvSandeshSend in state Idle 2025-01-20 Mon 01:51:57:243.381 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502793606912, Pid 77462]: SANDESH: Send FAILED: 1737337917242994 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-2650-2 process_status= [ [ [ module_id = contrail-dns instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], [ type = Database name = Cassandra server_addrs= [ [ (*_iter6) = 10.0.0.42:9041, (*_iter6) = 10.0.0.22:9041, (*_iter6) = 10.0.0.49:9041, ] ] status = Up description = Established Cassandra connection ], [ type = Database name = RabbitMQ server_addrs= [ [ (*_iter6) = 10.0.0.42:5673, (*_iter6) = 10.0.0.22:5673, (*_iter6) = 10.0.0.49:5673, ] ] status = Up description = RabbitMQ connection established ], ] ] flag_infos= [ [ ] ] description = Collector connection down ], ] ] ] 2025-01-20 Mon 01:51:57:243.388 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502793606912, Pid 77462]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-01-20 Mon 01:51:57:243.408 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502793606912, Pid 77462]: Processing scm::EvTcpDeleteSession in state Idle 2025-01-20 Mon 01:51:57:243.427 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502793606912, Pid 77462]: Processing scm::EvSandeshSend in state Idle 2025-01-20 Mon 01:51:57:243.647 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502793606912, Pid 77462]: SANDESH: Send FAILED: 1737337917243095 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-2650-2:Control:contrail-dns:0 client_info= [ status = Idle successful_connections = 5 pid = 77462 http_port = 8092 start_time = 1737337641318269 collector_name = collector_ip = 10.0.0.49:8086 collector_list= [ [ (*_iter40) = 10.0.0.42:8086, (*_iter40) = 10.0.0.49:8086, (*_iter40) = 10.0.0.22:8086, ] ] rx_socket_stats= [ bytes = 2034 calls = 4 average_bytes = 508 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 299610 calls = 27 average_bytes = 11096 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ConfigClientGetRowError [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 6 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigClientMgrDebug [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 711 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigurationMalformedReferenceWarningLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 17 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = DnsOperational [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 14 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapLinkOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 469 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapNodeOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 271 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 6 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 30 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlClientToServer [ messages_sent = 5 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlServerToClient [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 12 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 43 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 24 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveDnsInfo [ messages_sent = 2 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionCreate [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 6 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppCreateConnection [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 1520, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 73, _iter59->first = sent _iter59->second = 31, ] ] ] 2025-01-20 Mon 01:51:57:243.665 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502793606912, Pid 77462]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-01-20 Mon 01:51:57:312.837 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502768416512, Pid 77462]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 571 2025-01-20 Mon 01:51:57:338.742 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502772614912, Pid 77462]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 571 2025-01-20 Mon 01:51:57:369.756 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502793606912, Pid 77462]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 571 2025-01-20 Mon 01:51:57:497.573 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502764218112, Pid 77462]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 571 2025-01-20 Mon 01:51:57:516.820 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502785210112, Pid 77462]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 571 2025-01-20 Mon 01:51:57:532.635 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502785210112, Pid 77462]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 571 2025-01-20 Mon 01:51:57:666.178 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502764218112, Pid 77462]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 571 2025-01-20 Mon 01:51:57:695.776 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502785210112, Pid 77462]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 571 2025-01-20 Mon 01:51:57:707.805 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502785210112, Pid 77462]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 571 2025-01-20 Mon 01:51:57:869.441 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502768416512, Pid 77462]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 571 2025-01-20 Mon 01:51:57:895.272 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502789408512, Pid 77462]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 571 2025-01-20 Mon 01:51:57:923.426 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502747424512, Pid 77462]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 571 2025-01-20 Mon 01:51:58:041.264 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502781011712, Pid 77462]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 571 2025-01-20 Mon 01:51:58:077.485 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502793606912, Pid 77462]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 571 2025-01-20 Mon 01:51:58:108.588 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502760019712, Pid 77462]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 571 2025-01-20 Mon 01:51:58:206.114 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502772614912, Pid 77462]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 571 2025-01-20 Mon 01:51:58:247.731 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502764218112, Pid 77462]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 571 2025-01-20 Mon 01:51:58:275.104 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502755821312, Pid 77462]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 571 2025-01-20 Mon 01:51:58:375.074 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502772614912, Pid 77462]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 571 2025-01-20 Mon 01:51:59:658.723 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502953732224, Pid 77462]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.20.0.11:8092::10.20.0.18:45480 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 254 2025-01-20 Mon 01:51:59:714.222 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502785210112, Pid 77462]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 571 2025-01-20 Mon 01:51:59:745.294 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502785210112, Pid 77462]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 571 2025-01-20 Mon 01:51:59:784.349 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502793606912, Pid 77462]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 571 2025-01-20 Mon 01:52:00:946.981 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502953732224, Pid 77462]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.20.0.11:8092::10.20.0.11:41756 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 254 2025-01-20 Mon 01:52:01:100.208 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502953732224, Pid 77462]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.20.0.11:8092::10.20.0.7:53980 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 254 2025-01-20 Mon 01:52:08:559.221 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502747424512, Pid 77462]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 571 2025-01-20 Mon 01:52:08:578.312 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502760019712, Pid 77462]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 571 2025-01-20 Mon 01:52:09:051.865 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502793606912, Pid 77462]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 571 2025-01-20 Mon 01:52:19:710.305 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502760019712, Pid 77462]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 571 2025-01-20 Mon 01:52:19:762.268 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502781011712, Pid 77462]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 571 2025-01-20 Mon 01:52:19:794.334 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502793606912, Pid 77462]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 571 2025-01-20 Mon 01:52:24:541.786 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502797805312, Pid 77462]: Processing scm::EvSandeshSend in state Idle 2025-01-20 Mon 01:52:24:541.951 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502797805312, Pid 77462]: SANDESH: Send FAILED: 1737337944541644 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-2650-2:Control:contrail-dns:0 sm_queue_count = 0 sm_stats= [ ev_stats= [ [ [ event = scm::EvCollectorUpdate enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvConnectTimerExpired enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvIdleHoldTimerExpired enqueues = 7 dequeues = 7 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvSandeshMessageRecv enqueues = 2 dequeues = 2 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvSandeshSend enqueues = 107 dequeues = 107 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvStart enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpClose enqueues = 4 dequeues = 4 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpConnectFail enqueues = 5 dequeues = 5 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpConnected enqueues = 5 dequeues = 5 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpDeleteSession enqueues = 10 dequeues = 10 enqueue_fails = 0 dequeue_fails = 0 ], [ event = enqueues = 143 dequeues = 143 enqueue_fails = 0 dequeue_fails = 0 ], ] ] state = Idle state_since = 1737337917243044 last_state = Idle last_event = scm::EvSandeshSend last_event_at = 1737337917243426 ] max_sm_queue_count = 7 ] 2025-01-20 Mon 01:52:24:541.967 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502797805312, Pid 77462]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-01-20 Mon 01:52:24:541.976 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502797805312, Pid 77462]: Processing scm::EvSandeshSend in state Idle 2025-01-20 Mon 01:52:24:542.211 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502797805312, Pid 77462]: SANDESH: Send FAILED: 1737337944541759 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-2650-2:Control:contrail-dns:0 client_info= [ status = Idle successful_connections = 5 pid = 77462 http_port = 8092 start_time = 1737337641318269 collector_name = collector_ip = 10.0.0.49:8086 collector_list= [ [ (*_iter40) = 10.0.0.42:8086, (*_iter40) = 10.0.0.49:8086, (*_iter40) = 10.0.0.22:8086, ] ] rx_socket_stats= [ bytes = 2034 calls = 4 average_bytes = 508 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 299610 calls = 27 average_bytes = 11096 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ConfigClientGetRowError [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 6 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigClientMgrDebug [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 739 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigurationMalformedReferenceWarningLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 17 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = DnsOperational [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 14 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapLinkOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 469 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapNodeOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 271 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 6 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 31 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlClientToServer [ messages_sent = 5 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlServerToClient [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 12 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 45 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 27 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveDnsInfo [ messages_sent = 2 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionCreate [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 6 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppCreateConnection [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 1551, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 76, _iter59->first = sent _iter59->second = 31, ] ] ] 2025-01-20 Mon 01:52:24:542.235 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502797805312, Pid 77462]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-01-20 Mon 01:52:25:908.207 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502953732224, Pid 77462]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.20.0.11:8092::10.20.0.18:39588 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 254 2025-01-20 Mon 01:52:26:036.830 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502953732224, Pid 77462]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.20.0.11:8092::10.20.0.11:35314 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 254 2025-01-20 Mon 01:52:27:184.517 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502953732224, Pid 77462]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.20.0.11:8092::10.20.0.7:52376 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 254 2025-01-20 Mon 01:52:27:243.151 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502747424512, Pid 77462]: Processing scm::EvIdleHoldTimerExpired in state Idle 2025-01-20 Mon 01:52:27:243.366 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502747424512, Pid 77462]: Connect : Start Connect timer 10.0.0.49:8086 2025-01-20 Mon 01:52:27:243.498 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502747424512, Pid 77462]: Processing scm::EvSandeshSend in state Connect 2025-01-20 Mon 01:52:27:243.533 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502747424512, Pid 77462]: SANDESH: Send FAILED: 1737337947243231 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-2650-2 process_status= [ [ [ module_id = contrail-dns instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.49:8086, ] ] status = Initializing description = Idle : EvIdleHoldTimerExpired -> Connect ], [ type = Database name = Cassandra server_addrs= [ [ (*_iter6) = 10.0.0.42:9041, (*_iter6) = 10.0.0.22:9041, (*_iter6) = 10.0.0.49:9041, ] ] status = Up description = Established Cassandra connection ], [ type = Database name = RabbitMQ server_addrs= [ [ (*_iter6) = 10.0.0.42:5673, (*_iter6) = 10.0.0.22:5673, (*_iter6) = 10.0.0.49:5673, ] ] status = Up description = RabbitMQ connection established ], ] ] flag_infos= [ [ ] ] description = Collector connection down ], ] ] ] 2025-01-20 Mon 01:52:27:243.543 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502747424512, Pid 77462]: Wrong state: Connect for event: EvSandeshSend message: NodeStatusUVE 2025-01-20 Mon 01:52:27:243.553 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502747424512, Pid 77462]: Processing scm::EvSandeshSend in state Connect 2025-01-20 Mon 01:52:27:243.833 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502747424512, Pid 77462]: SANDESH: Send FAILED: 1737337947243480 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-2650-2:Control:contrail-dns:0 client_info= [ status = Connect successful_connections = 5 pid = 77462 http_port = 8092 start_time = 1737337641318269 collector_name = collector_ip = 10.0.0.49:8086 collector_list= [ [ (*_iter40) = 10.0.0.42:8086, (*_iter40) = 10.0.0.49:8086, (*_iter40) = 10.0.0.22:8086, ] ] rx_socket_stats= [ bytes = 2034 calls = 4 average_bytes = 508 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 299610 calls = 27 average_bytes = 11096 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ConfigClientGetRowError [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 6 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigClientMgrDebug [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 739 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigurationMalformedReferenceWarningLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 17 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = DnsOperational [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 14 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapLinkOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 469 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapNodeOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 271 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 6 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 31 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlClientToServer [ messages_sent = 5 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlServerToClient [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 12 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 47 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 30 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveDnsInfo [ messages_sent = 2 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionCreate [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 6 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppCreateConnection [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 1554, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 78, _iter59->first = sent _iter59->second = 31, ] ] ] 2025-01-20 Mon 01:52:27:243.864 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502747424512, Pid 77462]: Wrong state: Connect for event: EvSandeshSend message: SandeshModuleClientTrace 2025-01-20 Mon 01:52:27:245.638 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502953732224, Pid 77462]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.0.0.22:34918::10.0.0.49:8086(-1) < Active session connection complete src/contrail-common/io/tcp_session.cc 280 2025-01-20 Mon 01:52:27:245.669 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502953732224, Pid 77462]: 10.0.0.22:34918::10.0.0.49:8086(-1) OnSessionEvent TCP Connected 2025-01-20 Mon 01:52:27:245.677 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502953732224, Pid 77462]: 10.0.0.22:34918::10.0.0.49:8086(-1):EvTcpConnected 2025-01-20 Mon 01:52:27:245.713 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502793606912, Pid 77462]: Processing scm::EvTcpConnected in state Connect 2025-01-20 Mon 01:52:27:245.740 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502793606912, Pid 77462]: Connect : EvTcpConnected : Cancelling Connect timer 2025-01-20 Mon 01:52:27:245.783 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502793606912, Pid 77462]: ClientInit 2025-01-20 Mon 01:52:27:245.799 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502793606912, Pid 77462]: Sending Ctrl Message for cn-jenkins-deploy-platform-ansible-os-2650-2:contrail-dns:0:Control count 6 2025-01-20 Mon 01:52:27:245.903 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502793606912, Pid 77462]: Processing scm::EvSandeshSend in state ClientInit 2025-01-20 Mon 01:52:27:245.912 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502793606912, Pid 77462]: ClientInit : EvSandeshSend : NodeStatusUVE 2025-01-20 Mon 01:52:27:245.935 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502793606912, Pid 77462]: SANDESH: Send FAILED: 1737337947245776 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-2650-2 process_status= [ [ [ module_id = contrail-dns instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.49:8086, ] ] status = Initializing description = Connect : EvTcpConnected ], [ type = Database name = Cassandra server_addrs= [ [ (*_iter6) = 10.0.0.42:9041, (*_iter6) = 10.0.0.22:9041, (*_iter6) = 10.0.0.49:9041, ] ] status = Up description = Established Cassandra connection ], [ type = Database name = RabbitMQ server_addrs= [ [ (*_iter6) = 10.0.0.42:5673, (*_iter6) = 10.0.0.22:5673, (*_iter6) = 10.0.0.49:5673, ] ] status = Up description = RabbitMQ connection established ], ] ] flag_infos= [ [ ] ] description = Collector connection down ], ] ] ] 2025-01-20 Mon 01:52:27:245.951 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502793606912, Pid 77462]: Received UVE message in wrong state : NodeStatusUVE 2025-01-20 Mon 01:52:27:245.959 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502793606912, Pid 77462]: Processing scm::EvSandeshSend in state ClientInit 2025-01-20 Mon 01:52:27:245.965 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502793606912, Pid 77462]: ClientInit : EvSandeshSend : SandeshCtrlClientToServer 2025-01-20 Mon 01:52:27:245.975 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502793606912, Pid 77462]: Processing scm::EvSandeshSend in state ClientInit 2025-01-20 Mon 01:52:27:245.980 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502793606912, Pid 77462]: ClientInit : EvSandeshSend : SandeshModuleClientTrace 2025-01-20 Mon 01:52:27:246.205 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502793606912, Pid 77462]: SANDESH: Send FAILED: 1737337947245887 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-2650-2:Control:contrail-dns:0 client_info= [ status = ClientInit successful_connections = 6 pid = 77462 http_port = 8092 start_time = 1737337641318269 collector_name = collector_ip = 10.0.0.49:8086 collector_list= [ [ (*_iter40) = 10.0.0.42:8086, (*_iter40) = 10.0.0.49:8086, (*_iter40) = 10.0.0.22:8086, ] ] rx_socket_stats= [ bytes = 2034 calls = 4 average_bytes = 508 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 299610 calls = 27 average_bytes = 11096 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ConfigClientGetRowError [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 6 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigClientMgrDebug [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 739 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigurationMalformedReferenceWarningLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 17 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = DnsOperational [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 14 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapLinkOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 469 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapNodeOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 271 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 6 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 32 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlClientToServer [ messages_sent = 5 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlServerToClient [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 12 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 48 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 31 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveDnsInfo [ messages_sent = 2 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionCreate [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 6 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppCreateConnection [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 1555, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 80, _iter59->first = sent _iter59->second = 31, ] ] ] 2025-01-20 Mon 01:52:27:246.223 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502793606912, Pid 77462]: Received UVE message in wrong state : SandeshModuleClientTrace 2025-01-20 Mon 01:52:27:249.403 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502797805312, Pid 77462]: OnMessage control in state: ClientInit 2025-01-20 Mon 01:52:27:249.453 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502789408512, Pid 77462]: Processing scm::EvSandeshMessageRecv in state ClientInit 2025-01-20 Mon 01:52:27:249.475 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502789408512, Pid 77462]: ClientInit : EvSandeshMessageRecv 2025-01-20 Mon 01:52:27:249.503 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502789408512, Pid 77462]: Received Ctrl Message with size 0 2025-01-20 Mon 01:52:27:249.519 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502789408512, Pid 77462]: SyncUVE Syncing ObjectDns val name = cn-jenkins-deploy-platform-ansible-os-2650-2 start_time = 1737337641884778 build_info = {"build-info":[{"build-version":"nightly.r24.1.centos","build-time":"2025-01-16 00:14:43.877305","build-user":"root","build-hostname":"jenkins-build-prepare-sandbox-7545-1.novalocal","build-id":"nightly.r24.1.centos-nightly_r24.1_centos.el7@contrail","build-number":"unknown"}]} collector = 10.0.0.42:8086 self_ip_list= [ [ (*_iter12) = 10.20.0.11, ] ] proxy seq 1 2025-01-20 Mon 01:52:27:249.530 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502789408512, Pid 77462]: SyncAllMaps for DnsState:0 period 0 cycle 0 without seqno , total = 1 2025-01-20 Mon 01:52:27:249.834 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502789408512, Pid 77462]: SyncUVE Syncing ObjectGeneratorInfo val name = cn-jenkins-deploy-platform-ansible-os-2650-2:Control:contrail-dns:0 client_info= [ status = ClientInit successful_connections = 6 pid = 77462 http_port = 8092 start_time = 1737337641318269 collector_name = collector_ip = 10.0.0.49:8086 collector_list= [ [ (*_iter40) = 10.0.0.42:8086, (*_iter40) = 10.0.0.49:8086, (*_iter40) = 10.0.0.22:8086, ] ] rx_socket_stats= [ bytes = 2034 calls = 4 average_bytes = 508 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 299610 calls = 27 average_bytes = 11096 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] sm_queue_count = 0 sm_stats= [ ev_stats= [ [ [ event = scm::EvCollectorUpdate enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvConnectTimerExpired enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvIdleHoldTimerExpired enqueues = 7 dequeues = 7 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvSandeshMessageRecv enqueues = 2 dequeues = 2 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvSandeshSend enqueues = 107 dequeues = 107 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvStart enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpClose enqueues = 4 dequeues = 4 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpConnectFail enqueues = 5 dequeues = 5 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpConnected enqueues = 5 dequeues = 5 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpDeleteSession enqueues = 10 dequeues = 10 enqueue_fails = 0 dequeue_fails = 0 ], [ event = enqueues = 143 dequeues = 143 enqueue_fails = 0 dequeue_fails = 0 ], ] ] state = Idle state_since = 1737337917243044 last_state = Idle last_event = scm::EvSandeshSend last_event_at = 1737337917243426 ] session_stats= [ num_recv_msg = 0 num_recv_msg_fail = 0 num_recv_fail = 0 num_send_msg = 19 num_send_msg_fail = 0 num_send_buffer_fail = 0 num_wait_msgq_enqueue = 0 num_wait_msgq_dequeue = 0 num_write_ready_cb_error = 0 ] session_rx_socket_stats= [ bytes = 1017 calls = 2 average_bytes = 508 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] session_tx_socket_stats= [ bytes = 183601 calls = 16 average_bytes = 11475 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] msg_type_agg= [ [ _iter58->first = ConfigClientGetRowError [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 6 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigClientMgrDebug [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 739 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigurationMalformedReferenceWarningLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 17 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = DnsOperational [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 14 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapLinkOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 469 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapNodeOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 271 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 6 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 32 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlClientToServer [ messages_sent = 5 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshCtrlServerToClient [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 12 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 48 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 31 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveDnsInfo [ messages_sent = 2 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionCreate [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 6 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppCreateConnection [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 1555, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 80, _iter59->first = sent _iter59->second = 31, ] ] max_sm_queue_count = 7 proxy seq 60 2025-01-20 Mon 01:52:27:249.869 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502789408512, Pid 77462]: SyncAllMaps for ModuleClientState:0 period 0 cycle 0 without seqno , total = 1 2025-01-20 Mon 01:52:27:249.890 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502789408512, Pid 77462]: SyncUVE Syncing ObjectDns val name = cn-jenkins-deploy-platform-ansible-os-2650-2 process_status= [ [ [ module_id = contrail-dns instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.49:8086, ] ] status = Initializing description = Connect : EvTcpConnected ], [ type = Database name = Cassandra server_addrs= [ [ (*_iter6) = 10.0.0.42:9041, (*_iter6) = 10.0.0.22:9041, (*_iter6) = 10.0.0.49:9041, ] ] status = Up description = Established Cassandra connection ], [ type = Database name = RabbitMQ server_addrs= [ [ (*_iter6) = 10.0.0.42:5673, (*_iter6) = 10.0.0.22:5673, (*_iter6) = 10.0.0.49:5673, ] ] status = Up description = RabbitMQ connection established ], ] ] flag_infos= [ [ ] ] description = Collector connection down ], ] ] proxy seq 37 2025-01-20 Mon 01:52:27:249.902 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502789408512, Pid 77462]: SyncAllMaps for NodeStatus:0 period 0 cycle 0 without seqno , total = 1 2025-01-20 Mon 01:52:27:249.909 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502789408512, Pid 77462]: SyncAllMaps for PeerFlapData:0 period 0 cycle 0 without seqno , total = 0 2025-01-20 Mon 01:52:27:249.915 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502789408512, Pid 77462]: SyncAllMaps for PeerStatsData:0 period 0 cycle 0 without seqno , total = 0 2025-01-20 Mon 01:52:27:249.922 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502789408512, Pid 77462]: SyncAllMaps for XmppPeerInfoData:0 period 0 cycle 0 without seqno , total = 0 2025-01-20 Mon 01:52:27:249.950 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502789408512, Pid 77462]: Established 2025-01-20 Mon 01:52:37:127.654 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502776813312, Pid 77462]: OnMessage control in state: Established 2025-01-20 Mon 01:52:37:141.134 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502776813312, Pid 77462]: Processing scm::EvSandeshMessageRecv in state Established 2025-01-20 Mon 01:52:37:141.159 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502776813312, Pid 77462]: Established : EvSandeshMessageRecv 2025-01-20 Mon 01:52:37:141.265 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502776813312, Pid 77462]: Received Ctrl Message with size 3 2025-01-20 Mon 01:52:37:141.280 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502776813312, Pid 77462]: SyncAllMaps for DnsState:0 period 0 cycle 0 with seqno 1, total = 0 2025-01-20 Mon 01:52:37:141.290 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502776813312, Pid 77462]: SyncAllMaps for ModuleClientState:0 period 0 cycle 0 with seqno 61, total = 0 2025-01-20 Mon 01:52:37:141.299 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502776813312, Pid 77462]: SyncAllMaps for NodeStatus:0 period 0 cycle 0 with seqno 39, total = 0 2025-01-20 Mon 01:52:37:141.307 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502776813312, Pid 77462]: SyncAllMaps for PeerFlapData:0 period 0 cycle 0 without seqno , total = 0 2025-01-20 Mon 01:52:37:141.313 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502776813312, Pid 77462]: SyncAllMaps for PeerStatsData:0 period 0 cycle 0 without seqno , total = 0 2025-01-20 Mon 01:52:37:141.320 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502776813312, Pid 77462]: SyncAllMaps for XmppPeerInfoData:0 period 0 cycle 0 without seqno , total = 0 2025-01-20 Mon 01:53:34:003.253 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502768416512, Pid 77462]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 571 2025-01-20 Mon 01:53:44:584.686 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 140502797805312, Pid 77462]: DnsAgent [SYS_NOTICE]: DnsOperational: BIND named down; DNS is not operational controller/src/dns/mgr/dns_mgr.cc 747 2025-01-20 Mon 01:54:13:624.138 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139755028101248, Pid 106284]: DnsAgent [SYS_INFO]: VersionInfoLog: {"build-info":[{"build-version":"nightly","build-time":"2025-01-20 00:19:43.682579","build-user":"root","build-hostname":"jenkins-build-prepare-sandbox-7550-1.novalocal","build-id":"unknown","build-number":"unknown"}]} src/contrail-common/base/misc_utils.cc 41 2025-01-20 Mon 01:54:13:624.233 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139755028101248, Pid 106284]: SANDESH: ROLE : Generator 2025-01-20 Mon 01:54:13:624.242 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139755028101248, Pid 106284]: SANDESH: MODULE : contrail-dns 2025-01-20 Mon 01:54:13:624.248 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139755028101248, Pid 106284]: SANDESH: SOURCE : cn-jenkins-deploy-platform-ansible-os-2650-2 2025-01-20 Mon 01:54:13:624.252 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139755028101248, Pid 106284]: SANDESH: NODE TYPE : Control 2025-01-20 Mon 01:54:13:624.257 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139755028101248, Pid 106284]: SANDESH: INSTANCE ID : 0 2025-01-20 Mon 01:54:13:624.262 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139755028101248, Pid 106284]: SANDESH: HTTP SERVER PORT : 8092 2025-01-20 Mon 01:54:13:624.268 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139755028101248, Pid 106284]: SANDESH: System Log Send Rate Limit: 0 -> 0 2025-01-20 Mon 01:54:13:624.508 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139755028101248, Pid 106284]: HTTP Introspect Init 2025-01-20 Mon 01:54:13:624.616 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139755028101248, Pid 106284]: TCP [SYS_DEBUG]: TcpServerMessageLog: Server 0.0.0.0:8092 Initialization complete src/contrail-common/io/tcp_server.cc 137 2025-01-20 Mon 01:54:13:624.675 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139755028101248, Pid 106284]: Sandesh Http Server Port: 8092 2025-01-20 Mon 01:54:13:624.718 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139755028101248, Pid 106284]: SANDESH: NOT Writing http_port 8092TO : /tmp/contrail-dns.106161.http_port 2025-01-20 Mon 01:54:13:624.726 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139755028101248, Pid 106284]: SANDESH: CONNECT TO COLLECTOR: 1 2025-01-20 Mon 01:54:13:624.803 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139755028101248, Pid 106284]: Idle 2025-01-20 Mon 01:54:13:624.975 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139755028101248, Pid 106284]: SANDESH: No Client: 1737338053624891 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-2650-2:Control:contrail-dns:0 client_info= [ status = Idle successful_connections = 0 pid = 106284 http_port = 8092 start_time = 1737338053624812 collector_name = collector_ip = 0.0.0.0:0 collector_list= [ [ ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 0, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 1, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 0, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-01-20 Mon 01:54:13:625.113 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139755028101248, Pid 106284]: SANDESH: Logging: DISABLED -> ENABLED 2025-01-20 Mon 01:54:13:625.127 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139755028101248, Pid 106284]: SANDESH: Logging: LEVEL: [ INVALID ] -> [ SYS_DEBUG ] log4level: [ NOTSET ] -> [ DEBUG ] 2025-01-20 Mon 01:54:13:636.118 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754853242624, Pid 106284]: Processing scm::EvStart in state Idle 2025-01-20 Mon 01:54:13:636.173 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754853242624, Pid 106284]: Disconnect 2025-01-20 Mon 01:54:13:636.267 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754853242624, Pid 106284]: Processing scm::EvCollectorUpdate in state Disconnect 2025-01-20 Mon 01:54:13:636.469 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754853242624, Pid 106284]: Connect : Start Connect timer 10.0.0.42:8086 2025-01-20 Mon 01:54:13:636.537 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754853242624, Pid 106284]: Processing scm::EvSandeshSend in state Connect 2025-01-20 Mon 01:54:13:636.624 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754853242624, Pid 106284]: SANDESH: Send FAILED: 1737338053636251 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-2650-2:Control:contrail-dns:0 client_info= [ status = Disconnect successful_connections = 0 pid = 106284 http_port = 8092 start_time = 1737338053624812 collector_name = collector_ip = 0.0.0.0:0 collector_list= [ [ ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 1, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 0, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-01-20 Mon 01:54:13:636.643 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754853242624, Pid 106284]: Wrong state: Connect for event: EvSandeshSend message: SandeshModuleClientTrace 2025-01-20 Mon 01:54:13:636.652 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754853242624, Pid 106284]: Processing scm::EvSandeshSend in state Connect 2025-01-20 Mon 01:54:13:636.723 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754853242624, Pid 106284]: SANDESH: Send FAILED: 1737338053636320 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-2650-2:Control:contrail-dns:0 client_info= [ status = Disconnect successful_connections = 0 pid = 106284 http_port = 8092 start_time = 1737338053624812 collector_name = collector_ip = 10.0.0.42:8086 collector_list= [ [ (*_iter40) = 10.0.0.42:8086, (*_iter40) = 10.0.0.49:8086, (*_iter40) = 10.0.0.22:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 1, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 0, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-01-20 Mon 01:54:13:636.735 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754853242624, Pid 106284]: Wrong state: Connect for event: EvSandeshSend message: SandeshModuleClientTrace 2025-01-20 Mon 01:54:13:636.743 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754853242624, Pid 106284]: Processing scm::EvSandeshSend in state Connect 2025-01-20 Mon 01:54:13:636.804 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754853242624, Pid 106284]: SANDESH: Send FAILED: 1737338053636528 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-2650-2:Control:contrail-dns:0 client_info= [ status = Connect successful_connections = 0 pid = 106284 http_port = 8092 start_time = 1737338053624812 collector_name = collector_ip = 10.0.0.42:8086 collector_list= [ [ (*_iter40) = 10.0.0.42:8086, (*_iter40) = 10.0.0.49:8086, (*_iter40) = 10.0.0.22:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 1, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 0, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-01-20 Mon 01:54:13:636.813 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754853242624, Pid 106284]: Wrong state: Connect for event: EvSandeshSend message: SandeshModuleClientTrace 2025-01-20 Mon 01:54:13:779.884 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139755028101248, Pid 106284]: Applying named configuration failed 2025-01-20 Mon 01:54:13:780.263 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139755028101248, Pid 106284]: TCP [SYS_DEBUG]: TcpServerMessageLog: Server 10.20.0.11:8093 Initialization complete src/contrail-common/io/tcp_server.cc 137 2025-01-20 Mon 01:54:13:780.342 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139755028101248, Pid 106284]: DnsAgent [SYS_NOTICE]: DnsOperational: Bulk sync has been started controller/src/dns/main.cc 384 2025-01-20 Mon 01:54:13:780.431 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754849044224, Pid 106284]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Config Client Mgr SM: Start RabbitMqReader and init Database src/contrail-common/config-client-mgr/config_client_manager.cc 303 2025-01-20 Mon 01:54:13:780.540 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754849044224, Pid 106284]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: RabbitMQ SM: RabbitMQ connection down src/contrail-common/config-client-mgr/config_amqp_client.cc 158 2025-01-20 Mon 01:54:13:780.551 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754849044224, Pid 106284]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: RabbitMQ SM: Connect to Rabbit MQ with queue_delete TRUE src/contrail-common/config-client-mgr/config_amqp_client.cc 167 2025-01-20 Mon 01:54:13:780.692 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754857441024, Pid 106284]: Processing scm::EvSandeshSend in state Connect 2025-01-20 Mon 01:54:13:780.730 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754857441024, Pid 106284]: SANDESH: Send FAILED: 1737338053780516 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-2650-2 process_status= [ [ [ module_id = contrail-dns instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.42:8086, ] ] status = Initializing description = Disconnect : EvCollectorUpdate -> Connect ], [ type = Database name = RabbitMQ server_addrs= [ [ (*_iter6) = 10.0.0.42:5673, (*_iter6) = 10.0.0.22:5673, (*_iter6) = 10.0.0.49:5673, ] ] status = Down description = RabbitMQ connection down ], ] ] flag_infos= [ [ ] ] description = Number of connections:2, Expected:3 Missing: Database:Cassandra ], ] ] ] 2025-01-20 Mon 01:54:13:780.739 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754857441024, Pid 106284]: Wrong state: Connect for event: EvSandeshSend message: NodeStatusUVE 2025-01-20 Mon 01:54:13:900.950 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754849044224, Pid 106284]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: RabbitMQ SM: RabbitMQ connection established src/contrail-common/config-client-mgr/config_amqp_client.cc 151 2025-01-20 Mon 01:54:13:900.994 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754861639424, Pid 106284]: Processing scm::EvSandeshSend in state Connect 2025-01-20 Mon 01:54:13:901.053 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754861639424, Pid 106284]: SANDESH: Send FAILED: 1737338053900903 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-2650-2 process_status= [ [ [ module_id = contrail-dns instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.42:8086, ] ] status = Initializing description = Disconnect : EvCollectorUpdate -> Connect ], [ type = Database name = RabbitMQ server_addrs= [ [ (*_iter6) = 10.0.0.42:5673, (*_iter6) = 10.0.0.22:5673, (*_iter6) = 10.0.0.49:5673, ] ] status = Up description = RabbitMQ connection established ], ] ] flag_infos= [ [ ] ] description = Number of connections:2, Expected:3 Missing: Database:Cassandra ], ] ] ] 2025-01-20 Mon 01:54:13:901.064 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754861639424, Pid 106284]: Wrong state: Connect for event: EvSandeshSend message: NodeStatusUVE 2025-01-20 Mon 01:54:13:901.007 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754849044224, Pid 106284]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Config Client Mgr SM: Init Database src/contrail-common/config-client-mgr/config_client_manager.cc 307 2025-01-20 Mon 01:54:13:901.672 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754849044224, Pid 106284]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Lost Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 580 2025-01-20 Mon 01:54:13:901.683 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754849044224, Pid 106284]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Db Init src/contrail-common/config-client-mgr/config_cassandra_client.cc 87 2025-01-20 Mon 01:54:13:903.207 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754819655424, Pid 106284]: Processing scm::EvSandeshSend in state Connect 2025-01-20 Mon 01:54:13:903.248 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754819655424, Pid 106284]: SANDESH: Send FAILED: 1737338053901654 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-2650-2 process_status= [ [ [ module_id = contrail-dns instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.42:8086, ] ] status = Initializing description = Disconnect : EvCollectorUpdate -> Connect ], [ type = Database name = Cassandra server_addrs= [ [ (*_iter6) = 10.0.0.42:9041, (*_iter6) = 10.0.0.22:9041, (*_iter6) = 10.0.0.49:9041, ] ] status = Down description = Lost Cassandra connection ], [ type = Database name = RabbitMQ server_addrs= [ [ (*_iter6) = 10.0.0.42:5673, (*_iter6) = 10.0.0.22:5673, (*_iter6) = 10.0.0.49:5673, ] ] status = Up description = RabbitMQ connection established ], ] ] flag_infos= [ [ ] ] description = Collector, Database:Cassandra connection down ], ] ] ] 2025-01-20 Mon 01:54:13:903.257 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754819655424, Pid 106284]: Wrong state: Connect for event: EvSandeshSend message: NodeStatusUVE 2025-01-20 Mon 01:54:13:938.939 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754849044224, Pid 106284]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Db SetTableSpace src/contrail-common/config-client-mgr/config_cassandra_client.cc 95 2025-01-20 Mon 01:54:13:939.447 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754849044224, Pid 106284]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Db UseColumnFamily uuidTableName src/contrail-common/config-client-mgr/config_cassandra_client.cc 104 2025-01-20 Mon 01:54:13:939.478 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754849044224, Pid 106284]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Db UseColumnFamily fqnTableName src/contrail-common/config-client-mgr/config_cassandra_client.cc 110 2025-01-20 Mon 01:54:13:939.541 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754849044224, Pid 106284]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-01-20 Mon 01:54:13:939.551 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754849044224, Pid 106284]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: BulkDataSync Started src/contrail-common/config-client-mgr/config_cassandra_client.cc 393 2025-01-20 Mon 01:54:13:939.598 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754832250624, Pid 106284]: Processing scm::EvSandeshSend in state Connect 2025-01-20 Mon 01:54:13:939.650 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754832250624, Pid 106284]: SANDESH: Send FAILED: 1737338053939509 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-2650-2 process_status= [ [ [ module_id = contrail-dns instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.42:8086, ] ] status = Initializing description = Disconnect : EvCollectorUpdate -> Connect ], [ type = Database name = Cassandra server_addrs= [ [ (*_iter6) = 10.0.0.42:9041, (*_iter6) = 10.0.0.22:9041, (*_iter6) = 10.0.0.49:9041, ] ] status = Up description = Established Cassandra connection ], [ type = Database name = RabbitMQ server_addrs= [ [ (*_iter6) = 10.0.0.42:5673, (*_iter6) = 10.0.0.22:5673, (*_iter6) = 10.0.0.49:5673, ] ] status = Up description = RabbitMQ connection established ], ] ] flag_infos= [ [ ] ] description = Collector connection down ], ] ] ] 2025-01-20 Mon 01:54:13:939.672 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754832250624, Pid 106284]: Wrong state: Connect for event: EvSandeshSend message: NodeStatusUVE 2025-01-20 Mon 01:54:13:946.281 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754849044224, Pid 106284]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-01-20 Mon 01:54:13:948.339 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754849044224, Pid 106284]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-01-20 Mon 01:54:13:974.054 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754849044224, Pid 106284]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-01-20 Mon 01:54:13:976.931 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754849044224, Pid 106284]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-01-20 Mon 01:54:13:987.956 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754849044224, Pid 106284]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-01-20 Mon 01:54:13:990.153 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754849044224, Pid 106284]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-01-20 Mon 01:54:13:992.718 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754849044224, Pid 106284]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-01-20 Mon 01:54:13:994.893 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754849044224, Pid 106284]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-01-20 Mon 01:54:13:997.403 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754849044224, Pid 106284]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-01-20 Mon 01:54:14:001.739 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754849044224, Pid 106284]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-01-20 Mon 01:54:14:003.601 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754849044224, Pid 106284]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-01-20 Mon 01:54:14:006.689 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754849044224, Pid 106284]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-01-20 Mon 01:54:14:008.525 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754849044224, Pid 106284]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-01-20 Mon 01:54:14:010.472 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754849044224, Pid 106284]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-01-20 Mon 01:54:14:011.857 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754849044224, Pid 106284]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-01-20 Mon 01:54:14:013.518 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754849044224, Pid 106284]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-01-20 Mon 01:54:14:019.610 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754849044224, Pid 106284]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-01-20 Mon 01:54:14:024.107 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754849044224, Pid 106284]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-01-20 Mon 01:54:14:026.652 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754849044224, Pid 106284]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-01-20 Mon 01:54:14:029.226 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754849044224, Pid 106284]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-01-20 Mon 01:54:14:032.103 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754849044224, Pid 106284]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-01-20 Mon 01:54:14:033.652 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754849044224, Pid 106284]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-01-20 Mon 01:54:14:036.641 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754849044224, Pid 106284]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-01-20 Mon 01:54:14:037.980 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754849044224, Pid 106284]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-01-20 Mon 01:54:14:042.915 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754849044224, Pid 106284]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-01-20 Mon 01:54:14:044.760 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754849044224, Pid 106284]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-01-20 Mon 01:54:14:051.896 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754849044224, Pid 106284]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-01-20 Mon 01:54:14:061.064 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754849044224, Pid 106284]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-01-20 Mon 01:54:14:065.532 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754849044224, Pid 106284]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-01-20 Mon 01:54:14:069.211 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754849044224, Pid 106284]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-01-20 Mon 01:54:14:072.808 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754849044224, Pid 106284]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-01-20 Mon 01:54:14:074.549 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754849044224, Pid 106284]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-01-20 Mon 01:54:14:088.601 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754849044224, Pid 106284]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-01-20 Mon 01:54:14:091.973 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754849044224, Pid 106284]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-01-20 Mon 01:54:14:094.070 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754849044224, Pid 106284]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-01-20 Mon 01:54:14:099.269 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754849044224, Pid 106284]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-01-20 Mon 01:54:14:102.397 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754849044224, Pid 106284]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-01-20 Mon 01:54:14:103.312 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754849044224, Pid 106284]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-01-20 Mon 01:54:14:104.690 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754849044224, Pid 106284]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-01-20 Mon 01:54:14:105.874 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754849044224, Pid 106284]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-01-20 Mon 01:54:14:106.671 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754849044224, Pid 106284]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-01-20 Mon 01:54:14:108.181 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754849044224, Pid 106284]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-01-20 Mon 01:54:14:110.341 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754849044224, Pid 106284]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-01-20 Mon 01:54:14:128.339 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754849044224, Pid 106284]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-01-20 Mon 01:54:14:130.301 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754849044224, Pid 106284]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-01-20 Mon 01:54:14:145.840 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754849044224, Pid 106284]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-01-20 Mon 01:54:14:172.449 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754849044224, Pid 106284]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-01-20 Mon 01:54:14:176.537 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754849044224, Pid 106284]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-01-20 Mon 01:54:14:178.008 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754849044224, Pid 106284]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-01-20 Mon 01:54:14:184.323 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754849044224, Pid 106284]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-01-20 Mon 01:54:14:188.750 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754849044224, Pid 106284]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-01-20 Mon 01:54:14:190.914 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754849044224, Pid 106284]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-01-20 Mon 01:54:14:201.283 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754849044224, Pid 106284]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-01-20 Mon 01:54:14:237.566 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754849044224, Pid 106284]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-01-20 Mon 01:54:14:238.982 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754849044224, Pid 106284]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-01-20 Mon 01:54:14:239.972 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754849044224, Pid 106284]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-01-20 Mon 01:54:14:241.557 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754849044224, Pid 106284]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-01-20 Mon 01:54:14:242.717 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754849044224, Pid 106284]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-01-20 Mon 01:54:14:251.492 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754849044224, Pid 106284]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-01-20 Mon 01:54:14:253.209 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754849044224, Pid 106284]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-01-20 Mon 01:54:14:254.527 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754849044224, Pid 106284]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-01-20 Mon 01:54:14:261.956 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754849044224, Pid 106284]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-01-20 Mon 01:54:14:264.537 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754849044224, Pid 106284]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-01-20 Mon 01:54:14:266.390 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754849044224, Pid 106284]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-01-20 Mon 01:54:14:267.840 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754849044224, Pid 106284]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-01-20 Mon 01:54:14:270.552 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754849044224, Pid 106284]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-01-20 Mon 01:54:14:272.492 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754849044224, Pid 106284]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-01-20 Mon 01:54:14:273.472 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754849044224, Pid 106284]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-01-20 Mon 01:54:14:275.689 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754849044224, Pid 106284]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-01-20 Mon 01:54:14:278.453 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754849044224, Pid 106284]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-01-20 Mon 01:54:14:280.094 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754849044224, Pid 106284]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-01-20 Mon 01:54:14:282.018 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754849044224, Pid 106284]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-01-20 Mon 01:54:14:284.101 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754849044224, Pid 106284]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-01-20 Mon 01:54:14:298.011 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754849044224, Pid 106284]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-01-20 Mon 01:54:14:306.437 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754849044224, Pid 106284]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-01-20 Mon 01:54:14:308.120 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754849044224, Pid 106284]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-01-20 Mon 01:54:14:320.660 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754849044224, Pid 106284]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-01-20 Mon 01:54:14:333.732 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754849044224, Pid 106284]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-01-20 Mon 01:54:14:340.126 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754849044224, Pid 106284]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-01-20 Mon 01:54:14:365.072 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754849044224, Pid 106284]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-01-20 Mon 01:54:14:368.642 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754849044224, Pid 106284]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-01-20 Mon 01:54:14:370.563 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754849044224, Pid 106284]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-01-20 Mon 01:54:14:372.748 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754849044224, Pid 106284]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-01-20 Mon 01:54:14:387.771 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754849044224, Pid 106284]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-01-20 Mon 01:54:14:402.425 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754849044224, Pid 106284]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-01-20 Mon 01:54:14:413.648 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754849044224, Pid 106284]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-01-20 Mon 01:54:14:415.426 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754849044224, Pid 106284]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-01-20 Mon 01:54:14:418.229 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754849044224, Pid 106284]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-01-20 Mon 01:54:14:426.677 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754849044224, Pid 106284]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-01-20 Mon 01:54:14:428.416 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754849044224, Pid 106284]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-01-20 Mon 01:54:14:433.392 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754849044224, Pid 106284]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-01-20 Mon 01:54:14:440.976 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754849044224, Pid 106284]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-01-20 Mon 01:54:14:442.712 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754849044224, Pid 106284]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-01-20 Mon 01:54:14:446.577 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754849044224, Pid 106284]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-01-20 Mon 01:54:14:449.510 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754849044224, Pid 106284]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-01-20 Mon 01:54:14:451.149 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754849044224, Pid 106284]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-01-20 Mon 01:54:14:454.185 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754849044224, Pid 106284]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-01-20 Mon 01:54:14:456.061 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754849044224, Pid 106284]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-01-20 Mon 01:54:14:458.337 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754849044224, Pid 106284]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-01-20 Mon 01:54:14:459.998 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754849044224, Pid 106284]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-01-20 Mon 01:54:14:462.403 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754849044224, Pid 106284]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-01-20 Mon 01:54:14:471.331 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754849044224, Pid 106284]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-01-20 Mon 01:54:14:480.872 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754849044224, Pid 106284]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-01-20 Mon 01:54:14:490.614 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754849044224, Pid 106284]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-01-20 Mon 01:54:14:499.605 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754849044224, Pid 106284]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-01-20 Mon 01:54:14:504.191 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754849044224, Pid 106284]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-01-20 Mon 01:54:14:520.521 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754849044224, Pid 106284]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-01-20 Mon 01:54:14:532.013 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754849044224, Pid 106284]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-01-20 Mon 01:54:14:545.498 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754849044224, Pid 106284]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-01-20 Mon 01:54:14:555.748 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754849044224, Pid 106284]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-01-20 Mon 01:54:14:557.124 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754849044224, Pid 106284]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-01-20 Mon 01:54:14:558.617 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754849044224, Pid 106284]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-01-20 Mon 01:54:14:559.847 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754849044224, Pid 106284]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-01-20 Mon 01:54:14:562.261 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754849044224, Pid 106284]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-01-20 Mon 01:54:14:577.714 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754849044224, Pid 106284]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-01-20 Mon 01:54:14:586.241 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754849044224, Pid 106284]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-01-20 Mon 01:54:14:588.503 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754849044224, Pid 106284]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-01-20 Mon 01:54:14:594.130 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754849044224, Pid 106284]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-01-20 Mon 01:54:14:607.867 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754849044224, Pid 106284]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: One reader finished BulkSync src/contrail-common/config-client-mgr/config_cassandra_client.cc 558 2025-01-20 Mon 01:54:14:607.912 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754857441024, Pid 106284]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: One reader finished BulkSync src/contrail-common/config-client-mgr/config_cassandra_client.cc 558 2025-01-20 Mon 01:54:14:607.929 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754853242624, Pid 106284]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: One reader finished BulkSync src/contrail-common/config-client-mgr/config_cassandra_client.cc 558 2025-01-20 Mon 01:54:14:607.956 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754870036224, Pid 106284]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: One reader finished BulkSync src/contrail-common/config-client-mgr/config_cassandra_client.cc 558 2025-01-20 Mon 01:54:14:607.981 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754861639424, Pid 106284]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: One reader finished BulkSync src/contrail-common/config-client-mgr/config_cassandra_client.cc 558 2025-01-20 Mon 01:54:14:607.993 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754865837824, Pid 106284]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: One reader finished BulkSync src/contrail-common/config-client-mgr/config_cassandra_client.cc 558 2025-01-20 Mon 01:54:14:608.004 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754840647424, Pid 106284]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: One reader finished BulkSync src/contrail-common/config-client-mgr/config_cassandra_client.cc 558 2025-01-20 Mon 01:54:14:608.014 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754844845824, Pid 106284]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: BulkSyncDone by all readers src/contrail-common/config-client-mgr/config_cassandra_client.cc 554 2025-01-20 Mon 01:54:14:620.029 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754844845824, Pid 106284]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Config Client Mgr SM: End of RIB computed and notification sent src/contrail-common/config-client-mgr/config_client_manager.cc 197 2025-01-20 Mon 01:54:14:620.043 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754832250624, Pid 106284]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: TRUE src/contrail-common/config-client-mgr/config_client_manager.cc 225 2025-01-20 Mon 01:54:14:620.549 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754849044224, Pid 106284]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-01-20 Mon 01:54:14:624.106 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754832250624, Pid 106284]: Processing scm::EvSandeshSend in state Connect 2025-01-20 Mon 01:54:14:624.168 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754832250624, Pid 106284]: SANDESH: Send FAILED: 1737338054620097 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-2650-2 process_status= [ [ [ module_id = contrail-dns instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.42:8086, ] ] status = Initializing description = Disconnect : EvCollectorUpdate -> Connect ], [ type = Database name = Cassandra server_addrs= [ [ (*_iter6) = 10.0.0.42:9041, (*_iter6) = 10.0.0.22:9041, (*_iter6) = 10.0.0.49:9041, ] ] status = Up description = Established Cassandra connection ], [ type = Database name = RabbitMQ server_addrs= [ [ (*_iter6) = 10.0.0.42:5673, (*_iter6) = 10.0.0.22:5673, (*_iter6) = 10.0.0.49:5673, ] ] status = Up description = RabbitMQ connection established ], ] ] flag_infos= [ [ ] ] description = Collector connection down ], ] ] ] 2025-01-20 Mon 01:54:14:624.194 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754832250624, Pid 106284]: Wrong state: Connect for event: EvSandeshSend message: NodeStatusUVE 2025-01-20 Mon 01:54:14:625.491 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754849044224, Pid 106284]: IFMap [SYS_WARN]: ConfigurationMalformedReferenceWarningLog: bgp_peering No entry in metadata map bgp_router 83709758-e089-442e-8bc2-4932e76c585a controller/src/ifmap/client/config_json_parser.cc 223 2025-01-20 Mon 01:54:14:625.520 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754849044224, Pid 106284]: IFMap [SYS_WARN]: ConfigurationMalformedReferenceWarningLog: bgp_peering No entry in metadata map bgp_router 83709758-e089-442e-8bc2-4932e76c585a controller/src/ifmap/client/config_json_parser.cc 223 2025-01-20 Mon 01:54:14:625.831 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754828052224, Pid 106284]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating hardware:juniper-qfx5120-48y-8c controller/src/ifmap/ifmap_server_table.cc 95 2025-01-20 Mon 01:54:14:625.877 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754828052224, Pid 106284]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating alarm:default-global-system-config:system-defined-pending-cassandra-compaction-tasks controller/src/ifmap/ifmap_server_table.cc 95 2025-01-20 Mon 01:54:14:625.900 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754828052224, Pid 106284]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating global-system-config:default-global-system-config controller/src/ifmap/ifmap_server_table.cc 95 2025-01-20 Mon 01:54:14:625.923 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754828052224, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-alarm controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:625.940 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754828052224, Pid 106284]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating role-config:default-global-system-config:juniper-qfx5220:basic controller/src/ifmap/ifmap_server_table.cc 95 2025-01-20 Mon 01:54:14:625.956 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754828052224, Pid 106284]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating node-profile:default-global-system-config:juniper-qfx5220 controller/src/ifmap/ifmap_server_table.cc 95 2025-01-20 Mon 01:54:14:625.968 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754828052224, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating node-profile-role-config controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:625.983 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754828052224, Pid 106284]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating overlay-role:default-global-system-config:crb-gateway controller/src/ifmap/ifmap_server_table.cc 95 2025-01-20 Mon 01:54:14:626.002 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754828052224, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-overlay-role controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:626.029 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754828052224, Pid 106284]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating hardware:juniper-qfx5100-24q-aa controller/src/ifmap/ifmap_server_table.cc 95 2025-01-20 Mon 01:54:14:626.823 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754828052224, Pid 106284]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating job-template:default-global-system-config:show_config_template controller/src/ifmap/ifmap_server_table.cc 95 2025-01-20 Mon 01:54:14:626.886 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754828052224, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-job-template controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:626.902 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754828052224, Pid 106284]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating role-definition:default-global-system-config:lean-superspine controller/src/ifmap/ifmap_server_table.cc 95 2025-01-20 Mon 01:54:14:626.919 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754828052224, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-role-definition controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:626.936 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754828052224, Pid 106284]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating feature:default-global-system-config:infra-bms-access controller/src/ifmap/ifmap_server_table.cc 95 2025-01-20 Mon 01:54:14:626.950 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754828052224, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:626.983 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754832250624, Pid 106284]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating feature:default-global-system-config:underlay-ip-clos controller/src/ifmap/ifmap_server_table.cc 95 2025-01-20 Mon 01:54:14:626.995 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754832250624, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:627.009 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754832250624, Pid 106284]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating feature:default-global-system-config:telemetry controller/src/ifmap/ifmap_server_table.cc 95 2025-01-20 Mon 01:54:14:627.026 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754832250624, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:627.043 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754832250624, Pid 106284]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating overlay-role:default-global-system-config:lean controller/src/ifmap/ifmap_server_table.cc 95 2025-01-20 Mon 01:54:14:627.054 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754832250624, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-overlay-role controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:627.068 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754832250624, Pid 106284]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating physical-role:default-global-system-config:superspine controller/src/ifmap/ifmap_server_table.cc 95 2025-01-20 Mon 01:54:14:627.079 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754832250624, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-physical-role controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:627.093 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754832250624, Pid 106284]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating feature:default-global-system-config:vn-interconnect controller/src/ifmap/ifmap_server_table.cc 95 2025-01-20 Mon 01:54:14:627.118 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754832250624, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:627.131 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754832250624, Pid 106284]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating sflow-profile:default-domain:default-project:sflow-fabric-interfaces controller/src/ifmap/ifmap_server_table.cc 95 2025-01-20 Mon 01:54:14:627.147 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754832250624, Pid 106284]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating project:default-domain:default-project controller/src/ifmap/ifmap_server_table.cc 95 2025-01-20 Mon 01:54:14:627.158 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754832250624, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating project-sflow-profile controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:627.171 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754832250624, Pid 106284]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating job-template:default-global-system-config:hitless_upgrade_strategy_template controller/src/ifmap/ifmap_server_table.cc 95 2025-01-20 Mon 01:54:14:627.192 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754832250624, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-job-template controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:627.205 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754832250624, Pid 106284]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating config-node:default-global-system-config:cn-jenkins-deploy-platform-ansible-os-2650-1. controller/src/ifmap/ifmap_server_table.cc 95 2025-01-20 Mon 01:54:14:627.223 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754832250624, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-config-node controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:627.237 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754832250624, Pid 106284]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating telemetry-profile:default-domain:default-project:default-telemetry-profile-1 controller/src/ifmap/ifmap_server_table.cc 95 2025-01-20 Mon 01:54:14:627.263 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754832250624, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating project-telemetry-profile controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:627.280 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754832250624, Pid 106284]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating sflow-profile:default-domain:default-project:sflow-all-interfaces controller/src/ifmap/ifmap_server_table.cc 95 2025-01-20 Mon 01:54:14:627.293 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754832250624, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating telemetry-profile-sflow-profile controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:627.310 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754832250624, Pid 106284]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating hardware:juniper-mx80-48t controller/src/ifmap/ifmap_server_table.cc 95 2025-01-20 Mon 01:54:14:627.571 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754832250624, Pid 106284]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating config-properties:default-global-system-config:config_property controller/src/ifmap/ifmap_server_table.cc 95 2025-01-20 Mon 01:54:14:627.593 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754832250624, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-config-properties controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:627.611 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754832250624, Pid 106284]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating role-config:default-global-system-config:juniper-qfx10k:basic controller/src/ifmap/ifmap_server_table.cc 95 2025-01-20 Mon 01:54:14:627.631 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754832250624, Pid 106284]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating node-profile:default-global-system-config:juniper-qfx10k controller/src/ifmap/ifmap_server_table.cc 95 2025-01-20 Mon 01:54:14:627.649 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754832250624, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating node-profile-role-config controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:627.665 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754832250624, Pid 106284]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating alarm:default-global-system-config:system-defined-disk-usage-critical controller/src/ifmap/ifmap_server_table.cc 95 2025-01-20 Mon 01:54:14:627.684 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754832250624, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-alarm controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:627.697 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754832250624, Pid 106284]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating physical-role:default-global-system-config:pnf controller/src/ifmap/ifmap_server_table.cc 95 2025-01-20 Mon 01:54:14:627.712 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754832250624, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-physical-role controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:627.726 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754832250624, Pid 106284]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating job-template:default-global-system-config:show_mac_mob_template controller/src/ifmap/ifmap_server_table.cc 95 2025-01-20 Mon 01:54:14:627.747 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754832250624, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-job-template controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:627.759 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754832250624, Pid 106284]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating hardware:juniper-mx2010 controller/src/ifmap/ifmap_server_table.cc 95 2025-01-20 Mon 01:54:14:629.024 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754836449024, Pid 106284]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating service-appliance-set:default-global-system-config:opencontrail controller/src/ifmap/ifmap_server_table.cc 95 2025-01-20 Mon 01:54:14:629.063 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754836449024, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-service-appliance-set controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:629.081 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754836449024, Pid 106284]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating namespace:default-domain:default-namespace controller/src/ifmap/ifmap_server_table.cc 95 2025-01-20 Mon 01:54:14:629.095 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754836449024, Pid 106284]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating domain:default-domain controller/src/ifmap/ifmap_server_table.cc 95 2025-01-20 Mon 01:54:14:629.107 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754836449024, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating domain-namespace controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:629.129 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754836449024, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:629.144 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754836449024, Pid 106284]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating role-definition:default-global-system-config:rr-superspine controller/src/ifmap/ifmap_server_table.cc 95 2025-01-20 Mon 01:54:14:629.160 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754836449024, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-role-definition controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:629.178 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754836449024, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:629.196 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754836449024, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:629.211 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754836449024, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:629.225 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754836449024, Pid 106284]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating feature:default-global-system-config:overlay-bgp controller/src/ifmap/ifmap_server_table.cc 95 2025-01-20 Mon 01:54:14:629.236 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754836449024, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:629.252 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754836449024, Pid 106284]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating overlay-role:default-global-system-config:route-reflector controller/src/ifmap/ifmap_server_table.cc 95 2025-01-20 Mon 01:54:14:629.263 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754836449024, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-overlay-role controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:629.279 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754836449024, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-physical-role controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:629.292 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754836449024, Pid 106284]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating hardware:juniper-qfx5110-48s-4c controller/src/ifmap/ifmap_server_table.cc 95 2025-01-20 Mon 01:54:14:629.309 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754836449024, Pid 106284]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating hardware:juniper-vqfx-10000 controller/src/ifmap/ifmap_server_table.cc 95 2025-01-20 Mon 01:54:14:629.320 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754836449024, Pid 106284]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating overlay-role:default-global-system-config:erb-ucast-gateway controller/src/ifmap/ifmap_server_table.cc 95 2025-01-20 Mon 01:54:14:629.335 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754836449024, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-overlay-role controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:629.347 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754836449024, Pid 106284]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating device-functional-group:default-domain:default-project:Centrally-Routed-Border-Spine-With-Optimized-Multicast controller/src/ifmap/ifmap_server_table.cc 95 2025-01-20 Mon 01:54:14:629.368 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754836449024, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating project-device-functional-group controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:629.397 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754836449024, Pid 106284]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating physical-role:default-global-system-config:spine controller/src/ifmap/ifmap_server_table.cc 95 2025-01-20 Mon 01:54:14:629.410 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754836449024, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating device-functional-group-physical-role controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:629.430 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754836449024, Pid 106284]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating service-template:default-domain:netns-snat-template controller/src/ifmap/ifmap_server_table.cc 95 2025-01-20 Mon 01:54:14:629.448 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754836449024, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating domain-service-template controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:629.461 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754836449024, Pid 106284]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating virtual-machine-interface:default-global-system-config:an-jenkins-deploy-platform-ansible-os-2650-2:vhost0 controller/src/ifmap/ifmap_server_table.cc 95 2025-01-20 Mon 01:54:14:629.481 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754836449024, Pid 106284]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating virtual-router:default-global-system-config:an-jenkins-deploy-platform-ansible-os-2650-2 controller/src/ifmap/ifmap_server_table.cc 95 2025-01-20 Mon 01:54:14:629.493 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754836449024, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating virtual-router-virtual-machine-interface controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:629.510 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754836449024, Pid 106284]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating virtual-network:default-domain:default-project:ip-fabric controller/src/ifmap/ifmap_server_table.cc 95 2025-01-20 Mon 01:54:14:629.521 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754836449024, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating virtual-machine-interface-virtual-network controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:629.536 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754836449024, Pid 106284]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating policy-management:default-policy-management controller/src/ifmap/ifmap_server_table.cc 95 2025-01-20 Mon 01:54:14:629.551 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754836449024, Pid 106284]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating hardware:juniper-jnp10016 controller/src/ifmap/ifmap_server_table.cc 95 2025-01-20 Mon 01:54:14:629.561 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754836449024, Pid 106284]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating hardware:juniper-qfx5100-48th-afo controller/src/ifmap/ifmap_server_table.cc 95 2025-01-20 Mon 01:54:14:629.571 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754836449024, Pid 106284]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating role-definition:default-global-system-config:crb-access-spine controller/src/ifmap/ifmap_server_table.cc 95 2025-01-20 Mon 01:54:14:629.587 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754836449024, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-role-definition controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:629.605 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754836449024, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:629.619 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754836449024, Pid 106284]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating feature:default-global-system-config:firewall controller/src/ifmap/ifmap_server_table.cc 95 2025-01-20 Mon 01:54:14:629.630 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754836449024, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:629.647 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754836449024, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:629.660 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754836449024, Pid 106284]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating feature:default-global-system-config:port-profile controller/src/ifmap/ifmap_server_table.cc 95 2025-01-20 Mon 01:54:14:629.671 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754836449024, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:629.684 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754836449024, Pid 106284]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating feature:default-global-system-config:l2-gateway controller/src/ifmap/ifmap_server_table.cc 95 2025-01-20 Mon 01:54:14:629.696 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754836449024, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:629.711 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754836449024, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:629.725 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754836449024, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:629.738 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754836449024, Pid 106284]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating overlay-role:default-global-system-config:crb-access controller/src/ifmap/ifmap_server_table.cc 95 2025-01-20 Mon 01:54:14:629.750 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754836449024, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-overlay-role controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:629.768 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754836449024, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-physical-role controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:629.781 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754836449024, Pid 106284]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating alarm:default-global-system-config:system-defined-vrouter-interface controller/src/ifmap/ifmap_server_table.cc 95 2025-01-20 Mon 01:54:14:629.800 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754836449024, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-alarm controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:629.815 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754836449024, Pid 106284]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating role-definition:default-global-system-config:dci-gateway-leaf controller/src/ifmap/ifmap_server_table.cc 95 2025-01-20 Mon 01:54:14:629.829 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754836449024, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-role-definition controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:629.846 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754836449024, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:629.859 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754836449024, Pid 106284]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating feature:default-global-system-config:dc-gateway controller/src/ifmap/ifmap_server_table.cc 95 2025-01-20 Mon 01:54:14:629.872 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754836449024, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:629.887 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754836449024, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:629.901 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754836449024, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:629.915 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754836449024, Pid 106284]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating feature:default-global-system-config:l3-gateway controller/src/ifmap/ifmap_server_table.cc 95 2025-01-20 Mon 01:54:14:629.925 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754836449024, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:629.945 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754836449024, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:629.959 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754836449024, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:629.972 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754836449024, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:629.986 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754836449024, Pid 106284]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating overlay-role:default-global-system-config:dci-gateway controller/src/ifmap/ifmap_server_table.cc 95 2025-01-20 Mon 01:54:14:629.996 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754836449024, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-overlay-role controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:630.012 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754836449024, Pid 106284]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating physical-role:default-global-system-config:leaf controller/src/ifmap/ifmap_server_table.cc 95 2025-01-20 Mon 01:54:14:630.022 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754836449024, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-physical-role controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:641.557 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754857441024, Pid 106284]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-01-20 Mon 01:54:14:642.241 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754844845824, Pid 106284]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-01-20 Mon 01:54:14:643.274 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754844845824, Pid 106284]: IFMap [SYS_WARN]: ConfigurationMalformedReferenceWarningLog: bgp_peering No entry in metadata map bgp_router 182eed38-a088-41ce-8310-1c4f852a655f controller/src/ifmap/client/config_json_parser.cc 223 2025-01-20 Mon 01:54:14:643.291 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754844845824, Pid 106284]: IFMap [SYS_WARN]: ConfigurationMalformedReferenceWarningLog: bgp_peering No entry in metadata map bgp_router 182eed38-a088-41ce-8310-1c4f852a655f controller/src/ifmap/client/config_json_parser.cc 223 2025-01-20 Mon 01:54:14:643.652 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754844845824, Pid 106284]: IFMap [SYS_WARN]: ConfigurationMalformedReferenceWarningLog: instance_target No entry in metadata map routing_instance 4ffcf4e5-d53c-406d-827c-f4a48f0700ea controller/src/ifmap/client/config_json_parser.cc 223 2025-01-20 Mon 01:54:14:642.562 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754853242624, Pid 106284]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-01-20 Mon 01:54:14:642.607 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754861639424, Pid 106284]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-01-20 Mon 01:54:14:644.914 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754861639424, Pid 106284]: IFMap [SYS_WARN]: ConfigurationMalformedReferenceWarningLog: instance_target No entry in metadata map routing_instance 9daf18a3-d98c-4998-a14b-66920f4ab2ad controller/src/ifmap/client/config_json_parser.cc 223 2025-01-20 Mon 01:54:14:645.251 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754828052224, Pid 106284]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating job-template:default-global-system-config:show_interface_details_template controller/src/ifmap/ifmap_server_table.cc 95 2025-01-20 Mon 01:54:14:645.322 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754828052224, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-job-template controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:645.368 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754828052224, Pid 106284]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating role-definition:default-global-system-config:ar-replicator-spine controller/src/ifmap/ifmap_server_table.cc 95 2025-01-20 Mon 01:54:14:645.439 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754828052224, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-role-definition controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:645.534 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754828052224, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:645.582 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754828052224, Pid 106284]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating feature:default-global-system-config:assisted-replicator controller/src/ifmap/ifmap_server_table.cc 95 2025-01-20 Mon 01:54:14:645.611 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754828052224, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:645.663 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754828052224, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:645.692 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754828052224, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:645.714 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754828052224, Pid 106284]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating overlay-role:default-global-system-config:ar-replicator controller/src/ifmap/ifmap_server_table.cc 95 2025-01-20 Mon 01:54:14:645.763 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754828052224, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-overlay-role controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:645.805 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754828052224, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-physical-role controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:645.916 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754828052224, Pid 106284]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating role-definition:default-global-system-config:ar-client-leaf controller/src/ifmap/ifmap_server_table.cc 95 2025-01-20 Mon 01:54:14:645.939 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754828052224, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-role-definition controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:645.958 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754828052224, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:645.977 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754828052224, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:645.993 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754828052224, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:646.008 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754828052224, Pid 106284]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating overlay-role:default-global-system-config:ar-client controller/src/ifmap/ifmap_server_table.cc 95 2025-01-20 Mon 01:54:14:646.020 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754828052224, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-overlay-role controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:646.043 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754828052224, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-physical-role controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:646.069 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754828052224, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:646.084 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754828052224, Pid 106284]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating hardware:juniper-qfx10008 controller/src/ifmap/ifmap_server_table.cc 95 2025-01-20 Mon 01:54:14:646.104 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754828052224, Pid 106284]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating snmp-profile:default-domain:default-project:snmp-default-profile controller/src/ifmap/ifmap_server_table.cc 95 2025-01-20 Mon 01:54:14:646.125 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754828052224, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating project-snmp-profile controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:646.139 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754828052224, Pid 106284]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating hardware:juniper-qfx5100e-24q-2p controller/src/ifmap/ifmap_server_table.cc 95 2025-01-20 Mon 01:54:14:646.159 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754828052224, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:646.180 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754828052224, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating domain-project controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:646.194 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754828052224, Pid 106284]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating api-access-list:default-global-system-config:default-api-access-list controller/src/ifmap/ifmap_server_table.cc 95 2025-01-20 Mon 01:54:14:646.212 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754828052224, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-api-access-list controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:646.225 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754828052224, Pid 106284]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating config-database-node:default-global-system-config:cn-jenkins-deploy-platform-ansible-os-2650-3. controller/src/ifmap/ifmap_server_table.cc 95 2025-01-20 Mon 01:54:14:646.260 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754828052224, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-config-database-node controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:646.275 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754828052224, Pid 106284]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating telemetry-profile:default-domain:default-project:default-telemetry-profile-5 controller/src/ifmap/ifmap_server_table.cc 95 2025-01-20 Mon 01:54:14:646.293 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754828052224, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating project-telemetry-profile controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:646.311 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754828052224, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating telemetry-profile-snmp-profile controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:646.324 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754828052224, Pid 106284]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating database-node:default-global-system-config:cn-jenkins-deploy-platform-ansible-os-2650-3. controller/src/ifmap/ifmap_server_table.cc 95 2025-01-20 Mon 01:54:14:646.342 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754828052224, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-database-node controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:646.360 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754828052224, Pid 106284]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating role-definition:default-global-system-config:crb-mcast-gateway-leaf controller/src/ifmap/ifmap_server_table.cc 95 2025-01-20 Mon 01:54:14:646.390 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754828052224, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-role-definition controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:646.409 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754828052224, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:646.428 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754828052224, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:646.444 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754828052224, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:646.459 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754828052224, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:646.473 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754828052224, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:646.494 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754828052224, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:646.508 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754828052224, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:646.521 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754828052224, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:646.537 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754828052224, Pid 106284]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating overlay-role:default-global-system-config:crb-mcast-gateway controller/src/ifmap/ifmap_server_table.cc 95 2025-01-20 Mon 01:54:14:646.548 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754828052224, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-overlay-role controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:646.568 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754828052224, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-physical-role controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:646.580 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754828052224, Pid 106284]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating job-template:default-global-system-config:search_ip_mac_template controller/src/ifmap/ifmap_server_table.cc 95 2025-01-20 Mon 01:54:14:646.604 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754828052224, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-job-template controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:646.617 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754828052224, Pid 106284]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating access-control-list:default-domain:default-project:__no_rule__:ingress-access-control-list controller/src/ifmap/ifmap_server_table.cc 95 2025-01-20 Mon 01:54:14:646.636 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754828052224, Pid 106284]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating security-group:default-domain:default-project:__no_rule__ controller/src/ifmap/ifmap_server_table.cc 95 2025-01-20 Mon 01:54:14:646.647 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754828052224, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating security-group-access-control-list controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:646.661 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754828052224, Pid 106284]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating role-config:default-global-system-config:juniper-qfx5120:basic controller/src/ifmap/ifmap_server_table.cc 95 2025-01-20 Mon 01:54:14:646.676 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754828052224, Pid 106284]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating node-profile:default-global-system-config:juniper-qfx5120 controller/src/ifmap/ifmap_server_table.cc 95 2025-01-20 Mon 01:54:14:646.686 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754828052224, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating node-profile-role-config controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:646.698 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754828052224, Pid 106284]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating hardware:juniper-vsrx controller/src/ifmap/ifmap_server_table.cc 95 2025-01-20 Mon 01:54:14:646.712 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754828052224, Pid 106284]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating hardware:juniper-srx5800 controller/src/ifmap/ifmap_server_table.cc 95 2025-01-20 Mon 01:54:14:646.722 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754828052224, Pid 106284]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating hardware:juniper-qfx5100-48t controller/src/ifmap/ifmap_server_table.cc 95 2025-01-20 Mon 01:54:14:646.732 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754828052224, Pid 106284]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating feature:default-global-system-config:pnf-service-chaining controller/src/ifmap/ifmap_server_table.cc 95 2025-01-20 Mon 01:54:14:646.747 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754828052224, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:646.779 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754828052224, Pid 106284]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating alarm:default-global-system-config:system-defined-vrouter-limit-exceeded controller/src/ifmap/ifmap_server_table.cc 95 2025-01-20 Mon 01:54:14:646.797 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754828052224, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-alarm controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:646.819 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754828052224, Pid 106284]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating tag:label=fabric-overlay-loopback-ip controller/src/ifmap/ifmap_server_table.cc 95 2025-01-20 Mon 01:54:14:646.840 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754828052224, Pid 106284]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating tag-type:label controller/src/ifmap/ifmap_server_table.cc 95 2025-01-20 Mon 01:54:14:646.851 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754828052224, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating tag-tag-type controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:646.866 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754828052224, Pid 106284]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating device-functional-group:default-domain:default-project:L3-Gateway-Spine controller/src/ifmap/ifmap_server_table.cc 95 2025-01-20 Mon 01:54:14:646.884 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754828052224, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating project-device-functional-group controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:646.903 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754828052224, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating device-functional-group-physical-role controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:646.915 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754828052224, Pid 106284]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating tag-type:application controller/src/ifmap/ifmap_server_table.cc 95 2025-01-20 Mon 01:54:14:646.929 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754828052224, Pid 106284]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating analytics-alarm-node:default-global-system-config:cn-jenkins-deploy-platform-ansible-os-2650-1. controller/src/ifmap/ifmap_server_table.cc 95 2025-01-20 Mon 01:54:14:646.944 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754828052224, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-analytics-alarm-node controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:646.957 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754828052224, Pid 106284]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating tag:label=fabric-as-number controller/src/ifmap/ifmap_server_table.cc 95 2025-01-20 Mon 01:54:14:646.974 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754828052224, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating tag-tag-type controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:646.987 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754828052224, Pid 106284]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating hardware:juniper-srx5600 controller/src/ifmap/ifmap_server_table.cc 95 2025-01-20 Mon 01:54:14:646.998 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754828052224, Pid 106284]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating hardware:juniper-qfx5100e-48s-6q controller/src/ifmap/ifmap_server_table.cc 95 2025-01-20 Mon 01:54:14:647.008 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754828052224, Pid 106284]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating alarm:default-global-system-config:system-defined-process-connectivity controller/src/ifmap/ifmap_server_table.cc 95 2025-01-20 Mon 01:54:14:647.026 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754828052224, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-alarm controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:647.039 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754828052224, Pid 106284]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating role-definition:default-global-system-config:ar-replicator-leaf controller/src/ifmap/ifmap_server_table.cc 95 2025-01-20 Mon 01:54:14:647.053 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754828052224, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-role-definition controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:647.069 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754828052224, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:647.088 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754828052224, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:647.103 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754828052224, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:647.117 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754828052224, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:647.136 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754828052224, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-overlay-role controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:647.153 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754828052224, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-physical-role controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:647.165 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754828052224, Pid 106284]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating hardware:juniper-qfx5100e-96s-8q controller/src/ifmap/ifmap_server_table.cc 95 2025-01-20 Mon 01:54:14:647.176 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754828052224, Pid 106284]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating overlay-role:default-global-system-config:pnf-servicechain controller/src/ifmap/ifmap_server_table.cc 95 2025-01-20 Mon 01:54:14:647.192 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754828052224, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-overlay-role controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:647.205 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754828052224, Pid 106284]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating hardware:juniper-qfx5220-32cd controller/src/ifmap/ifmap_server_table.cc 95 2025-01-20 Mon 01:54:14:647.216 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754828052224, Pid 106284]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating analytics-snmp-node:default-global-system-config:cn-jenkins-deploy-platform-ansible-os-2650-3. controller/src/ifmap/ifmap_server_table.cc 95 2025-01-20 Mon 01:54:14:647.231 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754828052224, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-analytics-snmp-node controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:647.245 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754828052224, Pid 106284]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating virtual-network:default-domain:default-project:__link_local__ controller/src/ifmap/ifmap_server_table.cc 95 2025-01-20 Mon 01:54:14:647.275 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754828052224, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating project-virtual-network controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:647.288 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754828052224, Pid 106284]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating tag:label=fabric-loopback-ip controller/src/ifmap/ifmap_server_table.cc 95 2025-01-20 Mon 01:54:14:647.324 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754828052224, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating tag-tag-type controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:647.338 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754828052224, Pid 106284]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating alarm:default-global-system-config:system-defined-prouter-connectivity controller/src/ifmap/ifmap_server_table.cc 95 2025-01-20 Mon 01:54:14:647.354 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754828052224, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-alarm controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:647.367 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754828052224, Pid 106284]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating discovery-service-assignment:default-discovery-service-assignment controller/src/ifmap/ifmap_server_table.cc 95 2025-01-20 Mon 01:54:14:647.401 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754828052224, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:647.418 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754828052224, Pid 106284]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating role-definition:default-global-system-config:rr-leaf controller/src/ifmap/ifmap_server_table.cc 95 2025-01-20 Mon 01:54:14:647.433 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754828052224, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-role-definition controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:647.452 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754828052224, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:647.468 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754828052224, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:647.482 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754828052224, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:647.496 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754828052224, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:647.514 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754828052224, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-overlay-role controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:647.529 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754828052224, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-physical-role controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:647.542 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754828052224, Pid 106284]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating role-definition:default-global-system-config:lean-spine controller/src/ifmap/ifmap_server_table.cc 95 2025-01-20 Mon 01:54:14:647.559 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754828052224, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-role-definition controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:647.575 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754828052224, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:647.589 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754828052224, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:647.603 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754828052224, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:647.620 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754828052224, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-overlay-role controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:647.635 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754828052224, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-physical-role controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:647.647 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754828052224, Pid 106284]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating role-definition:default-global-system-config:rr-spine controller/src/ifmap/ifmap_server_table.cc 95 2025-01-20 Mon 01:54:14:647.663 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754828052224, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-role-definition controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:647.681 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754828052224, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:647.696 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754828052224, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:647.710 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754828052224, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:647.724 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754828052224, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:647.741 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754828052224, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-overlay-role controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:647.756 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754828052224, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-physical-role controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:647.768 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754828052224, Pid 106284]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating hardware:juniper-qfx5100-48th-afi controller/src/ifmap/ifmap_server_table.cc 95 2025-01-20 Mon 01:54:14:648.321 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754865837824, Pid 106284]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-01-20 Mon 01:54:14:648.730 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754865837824, Pid 106284]: IFMap [SYS_WARN]: ConfigurationMalformedReferenceWarningLog: overlay_role Link name is empty node_profile 18968228-aaec-4c03-8443-b0ef809a5b19 controller/src/ifmap/client/config_json_parser.cc 217 2025-01-20 Mon 01:54:14:649.029 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754865837824, Pid 106284]: IFMap [SYS_WARN]: ConfigurationMalformedReferenceWarningLog: bgp_peering No entry in metadata map bgp_router 666b6320-af65-4250-85a3-ffae7490bd3e controller/src/ifmap/client/config_json_parser.cc 223 2025-01-20 Mon 01:54:14:649.042 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754865837824, Pid 106284]: IFMap [SYS_WARN]: ConfigurationMalformedReferenceWarningLog: bgp_peering No entry in metadata map bgp_router 666b6320-af65-4250-85a3-ffae7490bd3e controller/src/ifmap/client/config_json_parser.cc 223 2025-01-20 Mon 01:54:14:650.325 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754857441024, Pid 106284]: IFMap [SYS_WARN]: ConfigurationMalformedReferenceWarningLog: instance_target No entry in metadata map routing_instance 9d8ae6b3-04bf-4e2d-9f8d-82f7b68168b7 controller/src/ifmap/client/config_json_parser.cc 223 2025-01-20 Mon 01:54:14:650.418 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754857441024, Pid 106284]: IFMap [SYS_WARN]: ConfigurationMalformedReferenceWarningLog: instance_target No entry in metadata map routing_instance a22df29f-de9c-4ba5-8202-c5cff385575f controller/src/ifmap/client/config_json_parser.cc 223 2025-01-20 Mon 01:54:14:651.848 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754844845824, Pid 106284]: IFMap [SYS_WARN]: ConfigurationMalformedReferenceWarningLog: overlay_role Link name is empty node_profile d21e436e-e28f-4769-9584-9728f9b6c801 controller/src/ifmap/client/config_json_parser.cc 217 2025-01-20 Mon 01:54:14:651.874 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754844845824, Pid 106284]: IFMap [SYS_WARN]: ConfigurationMalformedReferenceWarningLog: physical_role Link name is empty node_profile d21e436e-e28f-4769-9584-9728f9b6c801 controller/src/ifmap/client/config_json_parser.cc 217 2025-01-20 Mon 01:54:14:651.990 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754844845824, Pid 106284]: IFMap [SYS_WARN]: ConfigurationMalformedReferenceWarningLog: instance_target No entry in metadata map routing_instance e629c448-82cf-44a5-8d46-2670e37dbb61 controller/src/ifmap/client/config_json_parser.cc 223 2025-01-20 Mon 01:54:14:654.421 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754828052224, Pid 106284]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating hardware:juniper-mx960 controller/src/ifmap/ifmap_server_table.cc 95 2025-01-20 Mon 01:54:14:654.457 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754828052224, Pid 106284]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating job-template:default-global-system-config:role_assignment_template controller/src/ifmap/ifmap_server_table.cc 95 2025-01-20 Mon 01:54:14:654.487 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754828052224, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-job-template controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:654.520 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754828052224, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-overlay-role controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:654.538 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754828052224, Pid 106284]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating hardware:juniper-qfx10002-72q controller/src/ifmap/ifmap_server_table.cc 95 2025-01-20 Mon 01:54:14:654.551 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754828052224, Pid 106284]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating alarm:default-global-system-config:system-defined-bottle-request-size-limit controller/src/ifmap/ifmap_server_table.cc 95 2025-01-20 Mon 01:54:14:654.569 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754828052224, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-alarm controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:654.588 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754828052224, Pid 106284]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating service-template:default-domain:docker-template controller/src/ifmap/ifmap_server_table.cc 95 2025-01-20 Mon 01:54:14:654.608 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754828052224, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating domain-service-template controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:654.622 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754828052224, Pid 106284]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating device-functional-group:default-domain:default-project:L2-Server-Leaf controller/src/ifmap/ifmap_server_table.cc 95 2025-01-20 Mon 01:54:14:654.642 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754828052224, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating project-device-functional-group controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:654.663 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754828052224, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating device-functional-group-physical-role controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:654.676 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754828052224, Pid 106284]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating role-config:default-global-system-config:juniper-qfx5k-lean:basic controller/src/ifmap/ifmap_server_table.cc 95 2025-01-20 Mon 01:54:14:654.696 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754828052224, Pid 106284]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating node-profile:default-global-system-config:juniper-qfx5k-lean controller/src/ifmap/ifmap_server_table.cc 95 2025-01-20 Mon 01:54:14:654.707 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754828052224, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating node-profile-role-config controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:655.082 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754828052224, Pid 106284]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating hardware:juniper-mx240 controller/src/ifmap/ifmap_server_table.cc 95 2025-01-20 Mon 01:54:14:655.095 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754828052224, Pid 106284]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating role-config:default-global-system-config:juniper-qfx5k:basic controller/src/ifmap/ifmap_server_table.cc 95 2025-01-20 Mon 01:54:14:655.108 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754828052224, Pid 106284]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating node-profile:default-global-system-config:juniper-qfx5k controller/src/ifmap/ifmap_server_table.cc 95 2025-01-20 Mon 01:54:14:655.119 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754828052224, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating node-profile-role-config controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:655.135 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754828052224, Pid 106284]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating job-template:default-global-system-config:show_chassis_info_template controller/src/ifmap/ifmap_server_table.cc 95 2025-01-20 Mon 01:54:14:655.163 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754828052224, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-job-template controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:655.177 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754828052224, Pid 106284]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating alarm:default-global-system-config:system-defined-xmpp-close-reason controller/src/ifmap/ifmap_server_table.cc 95 2025-01-20 Mon 01:54:14:655.199 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754828052224, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-alarm controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:655.213 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754828052224, Pid 106284]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating hardware:juniper-qfx5200-48y controller/src/ifmap/ifmap_server_table.cc 95 2025-01-20 Mon 01:54:14:655.223 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754828052224, Pid 106284]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating hardware:juniper-mx480 controller/src/ifmap/ifmap_server_table.cc 95 2025-01-20 Mon 01:54:14:655.233 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754828052224, Pid 106284]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating job-template:default-global-system-config:fabric_config_template controller/src/ifmap/ifmap_server_table.cc 95 2025-01-20 Mon 01:54:14:655.256 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754828052224, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-job-template controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:655.269 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754828052224, Pid 106284]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating node-profile:default-global-system-config:juniper-mx controller/src/ifmap/ifmap_server_table.cc 95 2025-01-20 Mon 01:54:14:655.288 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754828052224, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-node-profile controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:655.311 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754828052224, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating node-profile-hardware controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:655.325 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754828052224, Pid 106284]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating hardware:juniper-jnp10008 controller/src/ifmap/ifmap_server_table.cc 95 2025-01-20 Mon 01:54:14:655.336 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754828052224, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating node-profile-hardware controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:655.350 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754828052224, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating node-profile-hardware controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:655.365 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754828052224, Pid 106284]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating hardware:juniper-mx204 controller/src/ifmap/ifmap_server_table.cc 95 2025-01-20 Mon 01:54:14:655.376 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754828052224, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating node-profile-hardware controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:655.402 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754828052224, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating node-profile-hardware controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:655.414 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754828052224, Pid 106284]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating hardware:juniper-vmx controller/src/ifmap/ifmap_server_table.cc 95 2025-01-20 Mon 01:54:14:655.424 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754828052224, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating node-profile-hardware controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:655.442 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754828052224, Pid 106284]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating hardware:juniper-mx2020 controller/src/ifmap/ifmap_server_table.cc 95 2025-01-20 Mon 01:54:14:655.452 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754828052224, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating node-profile-hardware controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:655.466 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754828052224, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating node-profile-hardware controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:655.478 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754828052224, Pid 106284]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating hardware:juniper-mx2008 controller/src/ifmap/ifmap_server_table.cc 95 2025-01-20 Mon 01:54:14:655.488 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754828052224, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating node-profile-hardware controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:655.505 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754828052224, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating node-profile-hardware controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:655.517 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754828052224, Pid 106284]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating hardware:juniper-mx10003 controller/src/ifmap/ifmap_server_table.cc 95 2025-01-20 Mon 01:54:14:655.527 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754828052224, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating node-profile-hardware controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:655.539 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754828052224, Pid 106284]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating hardware:juniper-mx80 controller/src/ifmap/ifmap_server_table.cc 95 2025-01-20 Mon 01:54:14:655.555 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754828052224, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating node-profile-hardware controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:655.569 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754828052224, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating node-profile-hardware controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:655.584 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754828052224, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating node-profile-job-template controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:655.596 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754828052224, Pid 106284]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating alarm:default-global-system-config:system-defined-package-version-mismatch controller/src/ifmap/ifmap_server_table.cc 95 2025-01-20 Mon 01:54:14:655.616 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754828052224, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-alarm controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:655.629 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754828052224, Pid 106284]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating config-database-node:default-global-system-config:cn-jenkins-deploy-platform-ansible-os-2650-1. controller/src/ifmap/ifmap_server_table.cc 95 2025-01-20 Mon 01:54:14:655.644 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754828052224, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-config-database-node controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:655.661 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754828052224, Pid 106284]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating alarm:default-global-system-config:system-defined-prouter-tsn-connectivity controller/src/ifmap/ifmap_server_table.cc 95 2025-01-20 Mon 01:54:14:655.678 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754828052224, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-alarm controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:655.696 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754828052224, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-overlay-role controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:655.712 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754828052224, Pid 106284]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating service-appliance-set:default-global-system-config:native controller/src/ifmap/ifmap_server_table.cc 95 2025-01-20 Mon 01:54:14:655.728 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754828052224, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-service-appliance-set controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:655.747 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754828052224, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-overlay-role controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:655.759 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754828052224, Pid 106284]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating analytics-node:default-global-system-config:cn-jenkins-deploy-platform-ansible-os-2650-3. controller/src/ifmap/ifmap_server_table.cc 95 2025-01-20 Mon 01:54:14:655.777 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754828052224, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-analytics-node controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:656.146 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754828052224, Pid 106284]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating hardware:juniper-qfx10002-36q controller/src/ifmap/ifmap_server_table.cc 95 2025-01-20 Mon 01:54:14:656.160 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754828052224, Pid 106284]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating netconf-profile:default-domain:default-project:netconf-default-profile controller/src/ifmap/ifmap_server_table.cc 95 2025-01-20 Mon 01:54:14:656.181 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754828052224, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating project-netconf-profile controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:656.197 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754828052224, Pid 106284]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating device-functional-group:default-domain:default-project:L3-Server-Leaf controller/src/ifmap/ifmap_server_table.cc 95 2025-01-20 Mon 01:54:14:656.215 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754828052224, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating project-device-functional-group controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:656.231 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754828052224, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating device-functional-group-physical-role controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:656.251 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754828052224, Pid 106284]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating overlay-role:default-global-system-config:collapsed-spine controller/src/ifmap/ifmap_server_table.cc 95 2025-01-20 Mon 01:54:14:656.266 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754828052224, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-overlay-role controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:656.280 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754828052224, Pid 106284]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating database-node:default-global-system-config:cn-jenkins-deploy-platform-ansible-os-2650-2. controller/src/ifmap/ifmap_server_table.cc 95 2025-01-20 Mon 01:54:14:656.298 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754828052224, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-database-node controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:656.319 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754828052224, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:656.331 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754828052224, Pid 106284]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating alarm:default-global-system-config:system-defined-core-files controller/src/ifmap/ifmap_server_table.cc 95 2025-01-20 Mon 01:54:14:656.349 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754828052224, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-alarm controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:656.362 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754828052224, Pid 106284]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating alarm:default-global-system-config:system-defined-disk-usage-high controller/src/ifmap/ifmap_server_table.cc 95 2025-01-20 Mon 01:54:14:656.393 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754828052224, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-alarm controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:656.407 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754828052224, Pid 106284]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating hardware:juniper-qfx5220-128c controller/src/ifmap/ifmap_server_table.cc 95 2025-01-20 Mon 01:54:14:656.424 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754828052224, Pid 106284]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating job-template:default-global-system-config:server_import_template controller/src/ifmap/ifmap_server_table.cc 95 2025-01-20 Mon 01:54:14:656.445 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754828052224, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-job-template controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:656.458 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754828052224, Pid 106284]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating job-template:default-global-system-config:show_ops_info_template controller/src/ifmap/ifmap_server_table.cc 95 2025-01-20 Mon 01:54:14:656.480 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754828052224, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-job-template controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:656.493 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754828052224, Pid 106284]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating alarm:default-global-system-config:system-defined-storage-cluster-state controller/src/ifmap/ifmap_server_table.cc 95 2025-01-20 Mon 01:54:14:656.510 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754828052224, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-alarm controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:656.523 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754828052224, Pid 106284]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating role-definition:default-global-system-config:dci-gateway-spine controller/src/ifmap/ifmap_server_table.cc 95 2025-01-20 Mon 01:54:14:656.542 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754828052224, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-role-definition controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:656.560 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754828052224, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:656.576 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754828052224, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:656.592 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754828052224, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:656.609 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754828052224, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:656.625 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754828052224, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:656.639 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754828052224, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:656.656 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754828052224, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:656.671 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754828052224, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:656.689 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754828052224, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-overlay-role controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:656.705 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754828052224, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-physical-role controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:656.722 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754828052224, Pid 106284]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating hardware:juniper-qfx5100-96s-8q controller/src/ifmap/ifmap_server_table.cc 95 2025-01-20 Mon 01:54:14:656.735 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754828052224, Pid 106284]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating tag-type:deployment controller/src/ifmap/ifmap_server_table.cc 95 2025-01-20 Mon 01:54:14:656.748 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754828052224, Pid 106284]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating hardware:juniper-qfx5200-32c-32q controller/src/ifmap/ifmap_server_table.cc 95 2025-01-20 Mon 01:54:14:656.758 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754828052224, Pid 106284]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating job-template:default-global-system-config:discover_device_template controller/src/ifmap/ifmap_server_table.cc 95 2025-01-20 Mon 01:54:14:656.782 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754828052224, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-job-template controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:656.795 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754828052224, Pid 106284]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating hardware:juniper-qfx5120-48t-6c controller/src/ifmap/ifmap_server_table.cc 95 2025-01-20 Mon 01:54:14:656.807 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754828052224, Pid 106284]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating telemetry-profile:default-domain:default-project:default-telemetry-profile-2 controller/src/ifmap/ifmap_server_table.cc 95 2025-01-20 Mon 01:54:14:656.826 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754828052224, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating project-telemetry-profile controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:656.844 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754828052224, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating telemetry-profile-sflow-profile controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:657.202 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754828052224, Pid 106284]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating tag-type:site controller/src/ifmap/ifmap_server_table.cc 95 2025-01-20 Mon 01:54:14:657.220 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754828052224, Pid 106284]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating alarm:default-global-system-config:system-defined-process-status controller/src/ifmap/ifmap_server_table.cc 95 2025-01-20 Mon 01:54:14:657.240 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754828052224, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-alarm controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:657.253 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754828052224, Pid 106284]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating hardware:juniper-srx4200 controller/src/ifmap/ifmap_server_table.cc 95 2025-01-20 Mon 01:54:14:657.264 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754828052224, Pid 106284]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating role-definition:default-global-system-config:crb-gateway-leaf controller/src/ifmap/ifmap_server_table.cc 95 2025-01-20 Mon 01:54:14:657.282 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754828052224, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-role-definition controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:657.309 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754828052224, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:657.324 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754828052224, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:657.338 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754828052224, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:657.355 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754828052224, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:657.369 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754828052224, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:657.393 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754828052224, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:657.407 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754828052224, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:657.425 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754828052224, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:657.439 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754828052224, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:657.454 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754828052224, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-overlay-role controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:657.472 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754828052224, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-physical-role controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:657.486 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754828052224, Pid 106284]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating role-definition:default-global-system-config:pnf-servicechain-spine controller/src/ifmap/ifmap_server_table.cc 95 2025-01-20 Mon 01:54:14:657.501 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754828052224, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-role-definition controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:657.516 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754828052224, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:657.534 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754828052224, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-overlay-role controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:657.549 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754828052224, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-physical-role controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:657.564 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754828052224, Pid 106284]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating hardware:juniper-srx4600 controller/src/ifmap/ifmap_server_table.cc 95 2025-01-20 Mon 01:54:14:657.578 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754828052224, Pid 106284]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating tag:label=fabric-pnf-servicechain-ip controller/src/ifmap/ifmap_server_table.cc 95 2025-01-20 Mon 01:54:14:657.597 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754828052224, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating tag-tag-type controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:657.611 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754828052224, Pid 106284]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating hardware:juniper-qfx5210-64c controller/src/ifmap/ifmap_server_table.cc 95 2025-01-20 Mon 01:54:14:657.622 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754828052224, Pid 106284]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating job-template:default-global-system-config:fabric_onboard_template controller/src/ifmap/ifmap_server_table.cc 95 2025-01-20 Mon 01:54:14:657.645 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754828052224, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-job-template controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:657.659 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754828052224, Pid 106284]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating device-functional-group:default-domain:default-project:L3-Storage-Leaf controller/src/ifmap/ifmap_server_table.cc 95 2025-01-20 Mon 01:54:14:657.675 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754828052224, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating project-device-functional-group controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:657.691 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754828052224, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating device-functional-group-physical-role controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:657.706 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754828052224, Pid 106284]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating sflow-profile:default-domain:default-project:sflow-access-interfaces controller/src/ifmap/ifmap_server_table.cc 95 2025-01-20 Mon 01:54:14:657.723 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754828052224, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating project-sflow-profile controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:657.736 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754828052224, Pid 106284]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating job-template:default-global-system-config:container_cleanup_template controller/src/ifmap/ifmap_server_table.cc 95 2025-01-20 Mon 01:54:14:657.762 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754828052224, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-job-template controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:657.776 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754828052224, Pid 106284]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating role-definition:default-global-system-config:crb-access-leaf controller/src/ifmap/ifmap_server_table.cc 95 2025-01-20 Mon 01:54:14:657.791 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754828052224, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-role-definition controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:657.806 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754828052224, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:657.828 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754828052224, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:657.843 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754828052224, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:657.857 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754828052224, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:657.871 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754828052224, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:657.887 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754828052224, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:657.901 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754828052224, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:658.257 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754828052224, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-overlay-role controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:658.276 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754828052224, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-physical-role controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:658.289 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754828052224, Pid 106284]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating tag-type:neutron_tag controller/src/ifmap/ifmap_server_table.cc 95 2025-01-20 Mon 01:54:14:658.302 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754828052224, Pid 106284]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating job-template:default-global-system-config:maintenance_mode_deactivate_template controller/src/ifmap/ifmap_server_table.cc 95 2025-01-20 Mon 01:54:14:658.322 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754828052224, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-job-template controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:658.337 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754828052224, Pid 106284]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating job-template:default-global-system-config:vcenter_import_template controller/src/ifmap/ifmap_server_table.cc 95 2025-01-20 Mon 01:54:14:658.360 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754828052224, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-job-template controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:658.373 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754828052224, Pid 106284]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating job-template:default-global-system-config:fabric_deletion_template controller/src/ifmap/ifmap_server_table.cc 95 2025-01-20 Mon 01:54:14:658.409 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754828052224, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-job-template controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:658.423 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754828052224, Pid 106284]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating job-template:default-global-system-config:discover_os_computes_template controller/src/ifmap/ifmap_server_table.cc 95 2025-01-20 Mon 01:54:14:658.443 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754828052224, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-job-template controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:658.461 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754828052224, Pid 106284]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating alarm:default-global-system-config:system-defined-xmpp-connectivity controller/src/ifmap/ifmap_server_table.cc 95 2025-01-20 Mon 01:54:14:658.478 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754828052224, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-alarm controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:658.492 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754828052224, Pid 106284]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating access-control-list:default-domain:default-project:__no_rule__:egress-access-control-list controller/src/ifmap/ifmap_server_table.cc 95 2025-01-20 Mon 01:54:14:658.510 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754828052224, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating security-group-access-control-list controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:658.526 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754828052224, Pid 106284]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating role-definition:default-global-system-config:crb-mcast-gateway-spine controller/src/ifmap/ifmap_server_table.cc 95 2025-01-20 Mon 01:54:14:658.539 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754828052224, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-role-definition controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:658.555 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754828052224, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:658.569 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754828052224, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:658.586 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754828052224, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:658.600 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754828052224, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:658.615 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754828052224, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:658.628 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754828052224, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:658.645 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754828052224, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:658.662 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754828052224, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:658.677 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754828052224, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-overlay-role controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:658.695 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754828052224, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-physical-role controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:658.711 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754828052224, Pid 106284]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating hardware:juniper-qfx5100-48sh-afo controller/src/ifmap/ifmap_server_table.cc 95 2025-01-20 Mon 01:54:14:658.730 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754828052224, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating project-security-group controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:658.746 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754828052224, Pid 106284]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating device-functional-group:default-domain:default-project:Border-Spine-in-Edge-Routed controller/src/ifmap/ifmap_server_table.cc 95 2025-01-20 Mon 01:54:14:658.765 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754828052224, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating project-device-functional-group controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:658.781 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754828052224, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating device-functional-group-physical-role controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:658.795 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754828052224, Pid 106284]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating analytics-snmp-node:default-global-system-config:cn-jenkins-deploy-platform-ansible-os-2650-2. controller/src/ifmap/ifmap_server_table.cc 95 2025-01-20 Mon 01:54:14:658.814 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754828052224, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-analytics-snmp-node controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:658.827 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754828052224, Pid 106284]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating telemetry-profile:default-domain:default-project:default-telemetry-profile-4 controller/src/ifmap/ifmap_server_table.cc 95 2025-01-20 Mon 01:54:14:658.843 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754828052224, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating project-telemetry-profile controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:658.858 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754828052224, Pid 106284]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating grpc-profile:default-domain:default-project:grpc-default-profile controller/src/ifmap/ifmap_server_table.cc 95 2025-01-20 Mon 01:54:14:658.872 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754828052224, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating telemetry-profile-grpc-profile controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:658.885 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754828052224, Pid 106284]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating alarm:default-global-system-config:system-defined-node-status controller/src/ifmap/ifmap_server_table.cc 95 2025-01-20 Mon 01:54:14:658.901 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754828052224, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-alarm controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:658.916 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754828052224, Pid 106284]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating hardware:juniper-qfx5120-32c controller/src/ifmap/ifmap_server_table.cc 95 2025-01-20 Mon 01:54:14:658.927 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754828052224, Pid 106284]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating telemetry-profile:default-domain:default-project:default-telemetry-profile-3 controller/src/ifmap/ifmap_server_table.cc 95 2025-01-20 Mon 01:54:14:658.941 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754828052224, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating project-telemetry-profile controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:658.960 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754828052224, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating telemetry-profile-sflow-profile controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:659.324 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754828052224, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-node-profile controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:659.341 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754828052224, Pid 106284]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating hardware:juniper-qfx5110-32q controller/src/ifmap/ifmap_server_table.cc 95 2025-01-20 Mon 01:54:14:659.351 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754828052224, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating node-profile-hardware controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:659.366 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754828052224, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating node-profile-hardware controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:659.395 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754828052224, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating node-profile-job-template controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:659.422 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754853242624, Pid 106284]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating device-functional-group:default-domain:default-project:L3-Server-Leaf-with-Optimized-Multicast controller/src/ifmap/ifmap_server_table.cc 95 2025-01-20 Mon 01:54:14:659.441 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754853242624, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating project-device-functional-group controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:659.457 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754853242624, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating device-functional-group-physical-role controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:659.476 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754853242624, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-overlay-role controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:659.493 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754853242624, Pid 106284]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating role-definition:default-global-system-config:pnf-servicechain-pnf controller/src/ifmap/ifmap_server_table.cc 95 2025-01-20 Mon 01:54:14:659.508 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754853242624, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-role-definition controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:659.524 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754853242624, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:659.540 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754853242624, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-overlay-role controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:659.555 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754853242624, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-physical-role controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:659.575 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754853242624, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-node-profile controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:659.594 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754853242624, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating node-profile-hardware controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:659.609 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754853242624, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating node-profile-hardware controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:659.623 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754853242624, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating node-profile-hardware controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:659.640 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754853242624, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating node-profile-job-template controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:659.654 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754853242624, Pid 106284]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating telemetry-profile:default-domain:default-project:default-telemetry-profile-6 controller/src/ifmap/ifmap_server_table.cc 95 2025-01-20 Mon 01:54:14:660.551 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754865837824, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating project-telemetry-profile controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:660.584 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754865837824, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating telemetry-profile-netconf-profile controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:660.619 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754865837824, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating project-virtual-network controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:660.634 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754865837824, Pid 106284]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating virtual-network:default-domain:default-project:default-virtual-network controller/src/ifmap/ifmap_server_table.cc 95 2025-01-20 Mon 01:54:14:660.650 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754865837824, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating project-virtual-network controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:660.669 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754865837824, Pid 106284]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating role-definition:default-global-system-config:dc-gateway-leaf controller/src/ifmap/ifmap_server_table.cc 95 2025-01-20 Mon 01:54:14:660.690 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754865837824, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-role-definition controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:660.709 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754865837824, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:660.725 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754865837824, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:660.741 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754865837824, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:660.757 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754865837824, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:660.783 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754865837824, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:660.798 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754865837824, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:660.813 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754865837824, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:660.829 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754865837824, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:660.846 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754865837824, Pid 106284]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating overlay-role:default-global-system-config:dc-gateway controller/src/ifmap/ifmap_server_table.cc 95 2025-01-20 Mon 01:54:14:660.858 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754865837824, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-overlay-role controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:660.874 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754865837824, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-physical-role controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:660.887 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754865837824, Pid 106284]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating virtual-network:default-domain:default-project:dci-network controller/src/ifmap/ifmap_server_table.cc 95 2025-01-20 Mon 01:54:14:660.913 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754865837824, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating project-virtual-network controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:668.619 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754870036224, Pid 106284]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-01-20 Mon 01:54:14:670.272 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754840647424, Pid 106284]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-01-20 Mon 01:54:14:672.110 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754865837824, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-physical-role controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:672.159 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754865837824, Pid 106284]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating role-definition:default-global-system-config:erb-leaf controller/src/ifmap/ifmap_server_table.cc 95 2025-01-20 Mon 01:54:14:672.184 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754865837824, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-role-definition controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:672.210 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754865837824, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:672.227 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754865837824, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:672.256 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754865837824, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:672.274 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754865837824, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:672.306 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754865837824, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:672.323 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754865837824, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:672.338 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754865837824, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:672.356 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754865837824, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:672.372 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754865837824, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:672.401 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754865837824, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-overlay-role controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:672.419 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754865837824, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-physical-role controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:672.444 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754865837824, Pid 106284]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating application-policy-set:default-policy-management:default-application-policy-set controller/src/ifmap/ifmap_server_table.cc 95 2025-01-20 Mon 01:54:14:672.464 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754865837824, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating policy-management-application-policy-set controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:672.483 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754865837824, Pid 106284]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating global-vrouter-config:default-global-system-config:default-global-vrouter-config controller/src/ifmap/ifmap_server_table.cc 95 2025-01-20 Mon 01:54:14:672.496 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754865837824, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating application-policy-set-global-vrouter-config controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:672.510 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754865837824, Pid 106284]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating network-ipam:default-domain:default-project:service-chain-flat-ipam controller/src/ifmap/ifmap_server_table.cc 95 2025-01-20 Mon 01:54:14:672.531 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754865837824, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating project-network-ipam controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:672.555 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754865837824, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating project-grpc-profile controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:672.570 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754865837824, Pid 106284]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating job-template:default-global-system-config:rma_activate_template controller/src/ifmap/ifmap_server_table.cc 95 2025-01-20 Mon 01:54:14:672.601 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754865837824, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-job-template controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:672.616 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754865837824, Pid 106284]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating analytics-snmp-node:default-global-system-config:cn-jenkins-deploy-platform-ansible-os-2650-1. controller/src/ifmap/ifmap_server_table.cc 95 2025-01-20 Mon 01:54:14:672.634 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754865837824, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-analytics-snmp-node controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:672.650 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754865837824, Pid 106284]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating node-profile:default-global-system-config:juniper-srx controller/src/ifmap/ifmap_server_table.cc 95 2025-01-20 Mon 01:54:14:672.669 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754865837824, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-node-profile controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:672.687 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754865837824, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating node-profile-hardware controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:672.703 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754865837824, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating node-profile-hardware controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:672.718 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754865837824, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating node-profile-hardware controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:672.732 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754865837824, Pid 106284]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating hardware:juniper-srx4100 controller/src/ifmap/ifmap_server_table.cc 95 2025-01-20 Mon 01:54:14:672.742 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754865837824, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating node-profile-hardware controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:672.755 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754865837824, Pid 106284]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating hardware:juniper-srx1500 controller/src/ifmap/ifmap_server_table.cc 95 2025-01-20 Mon 01:54:14:672.765 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754865837824, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating node-profile-hardware controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:672.781 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754865837824, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating node-profile-hardware controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:672.795 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754865837824, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating node-profile-hardware controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:672.808 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754865837824, Pid 106284]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating hardware:juniper-srx5400 controller/src/ifmap/ifmap_server_table.cc 95 2025-01-20 Mon 01:54:14:672.818 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754865837824, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating node-profile-hardware controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:672.834 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754865837824, Pid 106284]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating hardware:juniper-srx240h-poe controller/src/ifmap/ifmap_server_table.cc 95 2025-01-20 Mon 01:54:14:672.844 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754865837824, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating node-profile-hardware controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:672.860 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754865837824, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating node-profile-job-template controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:672.873 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754865837824, Pid 106284]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating job-template:default-global-system-config:image_upgrade_template controller/src/ifmap/ifmap_server_table.cc 95 2025-01-20 Mon 01:54:14:672.896 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754865837824, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-job-template controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:675.942 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754865837824, Pid 106284]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating job-template:default-global-system-config:existing_fabric_onboard_template controller/src/ifmap/ifmap_server_table.cc 95 2025-01-20 Mon 01:54:14:675.992 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754865837824, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-job-template controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:676.010 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754865837824, Pid 106284]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating role-definition:default-global-system-config:pnf-servicechain-leaf controller/src/ifmap/ifmap_server_table.cc 95 2025-01-20 Mon 01:54:14:676.029 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754865837824, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-role-definition controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:676.050 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754865837824, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:676.075 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754865837824, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-overlay-role controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:676.094 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754865837824, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-physical-role controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:676.109 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754865837824, Pid 106284]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating service-template:default-domain:nat-template controller/src/ifmap/ifmap_server_table.cc 95 2025-01-20 Mon 01:54:14:676.130 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754865837824, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating domain-service-template controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:676.145 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754865837824, Pid 106284]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating job-template:default-global-system-config:topology_discovery_template controller/src/ifmap/ifmap_server_table.cc 95 2025-01-20 Mon 01:54:14:676.167 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754865837824, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-job-template controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:676.201 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754865837824, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-physical-role controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:676.217 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754865837824, Pid 106284]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating alarm:default-global-system-config:system-defined-partial-sysinfo controller/src/ifmap/ifmap_server_table.cc 95 2025-01-20 Mon 01:54:14:676.237 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754865837824, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-alarm controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:676.251 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754865837824, Pid 106284]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating job-template:default-global-system-config:ztp_template controller/src/ifmap/ifmap_server_table.cc 95 2025-01-20 Mon 01:54:14:676.271 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754865837824, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-job-template controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:676.284 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754865837824, Pid 106284]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating job-template:default-global-system-config:cli_sync_template controller/src/ifmap/ifmap_server_table.cc 95 2025-01-20 Mon 01:54:14:676.303 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754865837824, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-job-template controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:676.319 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754865837824, Pid 106284]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating alarm:default-global-system-config:system-defined-bgp-connectivity controller/src/ifmap/ifmap_server_table.cc 95 2025-01-20 Mon 01:54:14:676.337 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754865837824, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-alarm controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:676.974 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754865837824, Pid 106284]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating device-functional-group:default-domain:default-project:Lean-Spine-with-Route-Reflector controller/src/ifmap/ifmap_server_table.cc 95 2025-01-20 Mon 01:54:14:676.995 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754865837824, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating project-device-functional-group controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:677.103 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754865837824, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating device-functional-group-physical-role controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:677.124 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754865837824, Pid 106284]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating feature-config:default-global-system-config:ar-client-spine:assisted-replicator controller/src/ifmap/ifmap_server_table.cc 95 2025-01-20 Mon 01:54:14:677.140 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754865837824, Pid 106284]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating role-definition:default-global-system-config:ar-client-spine controller/src/ifmap/ifmap_server_table.cc 95 2025-01-20 Mon 01:54:14:677.152 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754865837824, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature-config controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:677.164 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754865837824, Pid 106284]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating virtual-machine-interface:default-global-system-config:an-jenkins-deploy-platform-ansible-os-2650-1:vhost0 controller/src/ifmap/ifmap_server_table.cc 95 2025-01-20 Mon 01:54:14:677.185 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754865837824, Pid 106284]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating virtual-router:default-global-system-config:an-jenkins-deploy-platform-ansible-os-2650-1 controller/src/ifmap/ifmap_server_table.cc 95 2025-01-20 Mon 01:54:14:677.196 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754865837824, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating virtual-router-virtual-machine-interface controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:677.215 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754865837824, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating virtual-machine-interface-virtual-network controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:677.229 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754865837824, Pid 106284]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating alarm:default-global-system-config:system-defined-vrouter-table-limit-exceeded controller/src/ifmap/ifmap_server_table.cc 95 2025-01-20 Mon 01:54:14:677.246 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754865837824, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-alarm controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:677.258 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754865837824, Pid 106284]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating network-ipam:default-domain:default-project:default-network-ipam controller/src/ifmap/ifmap_server_table.cc 95 2025-01-20 Mon 01:54:14:677.274 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754865837824, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating project-network-ipam controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:677.287 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754865837824, Pid 106284]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating device-functional-group:default-domain:default-project:Centrally-Routed-Border-Spine controller/src/ifmap/ifmap_server_table.cc 95 2025-01-20 Mon 01:54:14:677.310 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754865837824, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating project-device-functional-group controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:677.327 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754865837824, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating device-functional-group-physical-role controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:677.341 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754865837824, Pid 106284]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating hardware:juniper-qfx5100-48sh-afi controller/src/ifmap/ifmap_server_table.cc 95 2025-01-20 Mon 01:54:14:677.360 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754865837824, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-overlay-role controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:677.374 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754865837824, Pid 106284]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating config-node:default-global-system-config:cn-jenkins-deploy-platform-ansible-os-2650-3. controller/src/ifmap/ifmap_server_table.cc 95 2025-01-20 Mon 01:54:14:677.401 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754865837824, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-config-node controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:678.024 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754865837824, Pid 106284]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating network-ipam:default-domain:default-project:_internal_ipam_ipv6_link_local controller/src/ifmap/ifmap_server_table.cc 95 2025-01-20 Mon 01:54:14:678.045 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754865837824, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating project-network-ipam controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:678.064 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754865837824, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-overlay-role controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:678.078 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754865837824, Pid 106284]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating hardware:juniper-qfx10002-60c controller/src/ifmap/ifmap_server_table.cc 95 2025-01-20 Mon 01:54:14:678.099 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754865837824, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-node-profile controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:678.117 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754865837824, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating node-profile-hardware controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:678.132 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754865837824, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating node-profile-hardware controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:678.152 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754865837824, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating node-profile-job-template controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:678.170 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754865837824, Pid 106284]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating job-template:default-global-system-config:discover_role_template controller/src/ifmap/ifmap_server_table.cc 95 2025-01-20 Mon 01:54:14:678.190 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754865837824, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-job-template controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:678.209 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754865837824, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:678.223 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754865837824, Pid 106284]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating node-profile:default-global-system-config:juniper-qfx10k-lean controller/src/ifmap/ifmap_server_table.cc 95 2025-01-20 Mon 01:54:14:678.239 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754865837824, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-node-profile controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:678.254 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754865837824, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating node-profile-job-template controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:678.267 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754865837824, Pid 106284]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating role-definition:default-global-system-config:dc-gateway-spine controller/src/ifmap/ifmap_server_table.cc 95 2025-01-20 Mon 01:54:14:678.282 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754865837824, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-role-definition controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:678.299 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754865837824, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:678.319 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754865837824, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:678.334 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754865837824, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:678.350 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754865837824, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:678.364 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754865837824, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:678.387 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754865837824, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:678.402 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754865837824, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:678.416 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754865837824, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:678.431 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754865837824, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-overlay-role controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:678.447 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754865837824, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-physical-role controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:678.461 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754865837824, Pid 106284]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating role-definition:default-global-system-config:crb-gateway-spine controller/src/ifmap/ifmap_server_table.cc 95 2025-01-20 Mon 01:54:14:681.253 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754865837824, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-role-definition controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:681.286 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754865837824, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:681.307 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754865837824, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:681.326 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754865837824, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:681.344 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754865837824, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:681.363 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754865837824, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:681.398 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754865837824, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:681.433 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754865837824, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:681.460 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754865837824, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:681.515 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754865837824, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:681.544 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754865837824, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-overlay-role controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:681.570 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754865837824, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-physical-role controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:681.592 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754865837824, Pid 106284]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating job-template:default-global-system-config:device_deletion_template controller/src/ifmap/ifmap_server_table.cc 95 2025-01-20 Mon 01:54:14:681.629 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754865837824, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-job-template controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:681.669 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754865837824, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-global-vrouter-config controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:681.692 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754865837824, Pid 106284]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating alarm:default-global-system-config:system-defined-conf-incorrect controller/src/ifmap/ifmap_server_table.cc 95 2025-01-20 Mon 01:54:14:681.725 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754865837824, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-alarm controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:681.744 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754865837824, Pid 106284]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating alarm:default-global-system-config:system-defined-conf-elements-inconsistency controller/src/ifmap/ifmap_server_table.cc 95 2025-01-20 Mon 01:54:14:681.771 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754865837824, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-alarm controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:681.806 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754865837824, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-node-profile controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:681.833 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754865837824, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating node-profile-hardware controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:681.863 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754865837824, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating node-profile-hardware controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:681.900 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754865837824, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating node-profile-hardware controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:681.922 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754865837824, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating node-profile-hardware controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:681.950 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754865837824, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating node-profile-hardware controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:681.970 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754865837824, Pid 106284]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating hardware:juniper-qfx5100e-48t-6q controller/src/ifmap/ifmap_server_table.cc 95 2025-01-20 Mon 01:54:14:681.990 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754865837824, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating node-profile-hardware controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:682.022 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754865837824, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating node-profile-hardware controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:682.048 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754865837824, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating node-profile-hardware controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:682.066 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754865837824, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating node-profile-hardware controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:682.085 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754865837824, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating node-profile-hardware controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:682.100 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754865837824, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating node-profile-hardware controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:682.116 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754865837824, Pid 106284]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating hardware:juniper-qfx5100-48s-6q controller/src/ifmap/ifmap_server_table.cc 95 2025-01-20 Mon 01:54:14:682.132 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754865837824, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating node-profile-hardware controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:682.148 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754865837824, Pid 106284]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating hardware:juniper-qfx5100-24q-2p controller/src/ifmap/ifmap_server_table.cc 95 2025-01-20 Mon 01:54:14:682.158 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754865837824, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating node-profile-hardware controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:682.173 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754865837824, Pid 106284]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating hardware:juniper-qfx5100-48t-6q controller/src/ifmap/ifmap_server_table.cc 95 2025-01-20 Mon 01:54:14:682.186 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754865837824, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating node-profile-hardware controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:682.203 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754865837824, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating node-profile-hardware controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:682.219 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754865837824, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating node-profile-hardware controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:682.234 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754865837824, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating node-profile-hardware controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:682.257 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754865837824, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating node-profile-job-template controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:682.273 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754865837824, Pid 106284]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating tag:label=fabric-management-ip controller/src/ifmap/ifmap_server_table.cc 95 2025-01-20 Mon 01:54:14:682.295 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754865837824, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating tag-tag-type controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:682.311 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754865837824, Pid 106284]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating analytics-node:default-global-system-config:cn-jenkins-deploy-platform-ansible-os-2650-1. controller/src/ifmap/ifmap_server_table.cc 95 2025-01-20 Mon 01:54:14:682.329 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754865837824, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-analytics-node controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:682.349 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754865837824, Pid 106284]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating service-template:default-domain:analyzer-template controller/src/ifmap/ifmap_server_table.cc 95 2025-01-20 Mon 01:54:14:682.388 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754865837824, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating domain-service-template controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:682.419 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754865837824, Pid 106284]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating role-definition:default-global-system-config:collapsed-spine controller/src/ifmap/ifmap_server_table.cc 95 2025-01-20 Mon 01:54:14:682.438 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754865837824, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-role-definition controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:682.459 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754865837824, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:682.480 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754865837824, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:682.497 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754865837824, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:682.513 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754865837824, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:682.529 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754865837824, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:682.544 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754865837824, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:682.561 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754865837824, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:682.581 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754865837824, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:682.597 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754865837824, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:682.615 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754865837824, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-overlay-role controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:682.633 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754865837824, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-physical-role controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:682.660 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754865837824, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating project-sflow-profile controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:682.681 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754865837824, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:682.696 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754865837824, Pid 106284]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating tag-type:tier controller/src/ifmap/ifmap_server_table.cc 95 2025-01-20 Mon 01:54:14:682.709 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754865837824, Pid 106284]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating access-control-list:default-domain:default-project:ip-fabric:ip-fabric controller/src/ifmap/ifmap_server_table.cc 95 2025-01-20 Mon 01:54:14:682.739 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754865837824, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating virtual-network-access-control-list controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:682.752 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754865837824, Pid 106284]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating alarm:default-global-system-config:system-defined-address-mismatch-compute controller/src/ifmap/ifmap_server_table.cc 95 2025-01-20 Mon 01:54:14:682.771 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754865837824, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-alarm controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:682.785 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754865837824, Pid 106284]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating fabric:default-global-system-config:default-fabric controller/src/ifmap/ifmap_server_table.cc 95 2025-01-20 Mon 01:54:14:682.802 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754865837824, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-fabric controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:682.815 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754865837824, Pid 106284]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating database-node:default-global-system-config:cn-jenkins-deploy-platform-ansible-os-2650-1. controller/src/ifmap/ifmap_server_table.cc 95 2025-01-20 Mon 01:54:14:682.830 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754865837824, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-database-node controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:682.849 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754865837824, Pid 106284]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating config-node:default-global-system-config:cn-jenkins-deploy-platform-ansible-os-2650-2. controller/src/ifmap/ifmap_server_table.cc 95 2025-01-20 Mon 01:54:14:682.868 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754865837824, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-config-node controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:682.884 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754865837824, Pid 106284]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating feature-config:default-global-system-config:ar-client-leaf:assisted-replicator controller/src/ifmap/ifmap_server_table.cc 95 2025-01-20 Mon 01:54:14:682.901 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754865837824, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature-config controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:682.922 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754865837824, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:682.945 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754865837824, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-virtual-router controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:682.964 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754865837824, Pid 106284]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating application-policy-set:default-domain:default-project:default-application-policy-set controller/src/ifmap/ifmap_server_table.cc 95 2025-01-20 Mon 01:54:14:683.117 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754861639424, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating project-application-policy-set controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:683.184 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754861639424, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-virtual-router controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:683.205 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754861639424, Pid 106284]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating tag-type:neutron_fwaas controller/src/ifmap/ifmap_server_table.cc 95 2025-01-20 Mon 01:54:14:683.220 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754861639424, Pid 106284]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating analytics-alarm-node:default-global-system-config:cn-jenkins-deploy-platform-ansible-os-2650-3. controller/src/ifmap/ifmap_server_table.cc 95 2025-01-20 Mon 01:54:14:683.257 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754861639424, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-analytics-alarm-node controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:683.275 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754861639424, Pid 106284]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating node-profile:default-global-system-config:device-functional-group controller/src/ifmap/ifmap_server_table.cc 95 2025-01-20 Mon 01:54:14:683.317 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754861639424, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-node-profile controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:683.342 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754861639424, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:683.358 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754861639424, Pid 106284]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating config-database-node:default-global-system-config:cn-jenkins-deploy-platform-ansible-os-2650-2. controller/src/ifmap/ifmap_server_table.cc 95 2025-01-20 Mon 01:54:14:683.414 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754861639424, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-config-database-node controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:683.459 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754861639424, Pid 106284]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating job-template:default-global-system-config:maintenance_mode_activate_template controller/src/ifmap/ifmap_server_table.cc 95 2025-01-20 Mon 01:54:14:683.501 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754861639424, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-job-template controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:684.219 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754861639424, Pid 106284]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating tag:label=fabric-peer-ip controller/src/ifmap/ifmap_server_table.cc 95 2025-01-20 Mon 01:54:14:684.246 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754861639424, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating tag-tag-type controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:684.284 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754861639424, Pid 106284]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating role-config:default-global-system-config:juniper-qfx10k-lean:basic controller/src/ifmap/ifmap_server_table.cc 95 2025-01-20 Mon 01:54:14:684.313 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754861639424, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating node-profile-role-config controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:684.331 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754861639424, Pid 106284]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating service-template:default-domain:haproxy-loadbalancer-template controller/src/ifmap/ifmap_server_table.cc 95 2025-01-20 Mon 01:54:14:684.390 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754861639424, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating domain-service-template controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:684.411 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754861639424, Pid 106284]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating virtual-network:default-domain:default-project:_internal_vn_ipv6_link_local controller/src/ifmap/ifmap_server_table.cc 95 2025-01-20 Mon 01:54:14:684.454 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754861639424, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating project-virtual-network controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:684.475 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754861639424, Pid 106284]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating virtual-network-network-ipam:attr(default-domain:default-project:_internal_ipam_ipv6_link_local,default-domain:default-project:_internal_vn_ipv6_link_local) controller/src/ifmap/ifmap_server_table.cc 95 2025-01-20 Mon 01:54:14:684.488 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754861639424, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating virtual-network-network-ipam controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:684.504 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754861639424, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating virtual-network-network-ipam controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:684.540 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754861639424, Pid 106284]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating job-template:default-global-system-config:hardware_inventory_template controller/src/ifmap/ifmap_server_table.cc 95 2025-01-20 Mon 01:54:14:684.586 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754832250624, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-job-template controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:684.635 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754832250624, Pid 106284]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating job-template:default-global-system-config:test_overlay_connectivity_template controller/src/ifmap/ifmap_server_table.cc 95 2025-01-20 Mon 01:54:14:684.677 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754832250624, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-job-template controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:684.693 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754832250624, Pid 106284]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating job-template:default-global-system-config:node_profile_template controller/src/ifmap/ifmap_server_table.cc 95 2025-01-20 Mon 01:54:14:684.716 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754832250624, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-job-template controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:684.756 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754832250624, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-physical-role controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:684.774 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754832250624, Pid 106284]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating job-template:default-global-system-config:check_multicast_template controller/src/ifmap/ifmap_server_table.cc 95 2025-01-20 Mon 01:54:14:684.847 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754832250624, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-job-template controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:684.886 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754832250624, Pid 106284]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating role-config:default-global-system-config:juniper-srx:basic controller/src/ifmap/ifmap_server_table.cc 95 2025-01-20 Mon 01:54:14:684.907 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754832250624, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating node-profile-role-config controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:684.931 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754832250624, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:684.965 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754832250624, Pid 106284]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating feature-config:default-global-system-config:erb-leaf:l3-gateway controller/src/ifmap/ifmap_server_table.cc 95 2025-01-20 Mon 01:54:14:684.986 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754832250624, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature-config controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:685.000 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754832250624, Pid 106284]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating device-functional-group:default-domain:default-project:Border-Leaf-in-Edge-Routed controller/src/ifmap/ifmap_server_table.cc 95 2025-01-20 Mon 01:54:14:685.038 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754832250624, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating project-device-functional-group controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:685.059 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754832250624, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating device-functional-group-physical-role controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:685.079 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754832250624, Pid 106284]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating role-config:default-global-system-config:juniper-mx:basic controller/src/ifmap/ifmap_server_table.cc 95 2025-01-20 Mon 01:54:14:685.136 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754832250624, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating node-profile-role-config controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:685.152 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754832250624, Pid 106284]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating tag:label=fabric-ebgp-as-number controller/src/ifmap/ifmap_server_table.cc 95 2025-01-20 Mon 01:54:14:685.189 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754832250624, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating tag-tag-type controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:685.205 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754832250624, Pid 106284]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating alarm:default-global-system-config:system-defined-vrouter-l3mh-physical-interface-link-down controller/src/ifmap/ifmap_server_table.cc 95 2025-01-20 Mon 01:54:14:685.227 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754832250624, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-alarm controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:685.244 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754832250624, Pid 106284]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating global-qos-config:default-global-system-config:default-global-qos-config controller/src/ifmap/ifmap_server_table.cc 95 2025-01-20 Mon 01:54:14:685.259 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754832250624, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-global-qos-config controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:685.275 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754832250624, Pid 106284]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating alarm:default-global-system-config:system-defined-address-mismatch-control controller/src/ifmap/ifmap_server_table.cc 95 2025-01-20 Mon 01:54:14:685.291 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754832250624, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-alarm controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:685.308 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754832250624, Pid 106284]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating hardware:juniper-qfx10016 controller/src/ifmap/ifmap_server_table.cc 95 2025-01-20 Mon 01:54:14:685.332 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754832250624, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-node-profile controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:685.351 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754832250624, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating node-profile-hardware controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:685.369 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754832250624, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating node-profile-hardware controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:685.402 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754832250624, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating node-profile-hardware controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:685.432 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754832250624, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating node-profile-hardware controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:685.448 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754832250624, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating node-profile-hardware controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:685.467 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754832250624, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating node-profile-hardware controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:685.483 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754832250624, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating node-profile-job-template controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:685.506 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754832250624, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-role-definition controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:685.525 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754832250624, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:685.541 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754832250624, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:685.563 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754832250624, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:685.580 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754832250624, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-overlay-role controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:685.596 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754832250624, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-physical-role controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:685.620 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754832250624, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-overlay-role controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:685.638 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754832250624, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-overlay-role controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:685.652 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754832250624, Pid 106284]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating analytics-node:default-global-system-config:cn-jenkins-deploy-platform-ansible-os-2650-2. controller/src/ifmap/ifmap_server_table.cc 95 2025-01-20 Mon 01:54:14:685.669 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754832250624, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-analytics-node controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:685.685 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754832250624, Pid 106284]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating job-template:default-global-system-config:device_import_template controller/src/ifmap/ifmap_server_table.cc 95 2025-01-20 Mon 01:54:14:685.707 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754832250624, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-job-template controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:685.721 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754832250624, Pid 106284]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating job-template:default-global-system-config:discover_server_template controller/src/ifmap/ifmap_server_table.cc 95 2025-01-20 Mon 01:54:14:685.743 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754832250624, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-job-template controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:685.759 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754832250624, Pid 106284]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating analytics-alarm-node:default-global-system-config:cn-jenkins-deploy-platform-ansible-os-2650-2. controller/src/ifmap/ifmap_server_table.cc 95 2025-01-20 Mon 01:54:14:685.774 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754832250624, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-analytics-alarm-node controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 01:54:14:686.965 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139755028101248, Pid 106284]: DnsAgent [SYS_NOTICE]: DnsOperational: EndOfRib point was reached, IF-MAP objects have been created controller/src/dns/main.cc 86 2025-01-20 Mon 01:54:14:686.993 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139755028101248, Pid 106284]: DnsAgent [SYS_NOTICE]: DnsOperational: The IF-MAP database table has been updated: __ifmap__.virtual_DNS.0, counting 0 elements controller/src/dns/main.cc 98 2025-01-20 Mon 01:54:14:687.019 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139755028101248, Pid 106284]: DnsAgent [SYS_NOTICE]: DnsOperational: The IF-MAP database table has been updated: __ifmap__.virtual_DNS_record.0, counting 0 elements controller/src/dns/main.cc 98 2025-01-20 Mon 01:54:14:687.028 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139755028101248, Pid 106284]: DnsAgent [SYS_NOTICE]: DnsOperational: The IF-MAP database table has been updated: __ifmap__.network_ipam.0, counting 3 elements controller/src/dns/main.cc 98 2025-01-20 Mon 01:54:14:687.037 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139755028101248, Pid 106284]: DnsAgent [SYS_NOTICE]: DnsOperational: The IF-MAP database table has been updated: __ifmap__.virtual_network_network_ipam.0, counting 1 elements controller/src/dns/main.cc 98 2025-01-20 Mon 01:54:14:687.046 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139755028101248, Pid 106284]: DnsAgent [SYS_NOTICE]: DnsOperational: The IF-MAP database table has been updated: __ifmap__.global_qos_config.0, counting 1 elements controller/src/dns/main.cc 98 2025-01-20 Mon 01:54:14:687.054 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139755028101248, Pid 106284]: DnsAgent [SYS_NOTICE]: DnsOperational: VnniConfig has been updated after the bulk sync, the number of elements: 1 controller/src/dns/main.cc 109 2025-01-20 Mon 01:54:14:687.062 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139755028101248, Pid 106284]: DnsAgent [SYS_NOTICE]: DnsOperational: IpamConfig has been updated after the bulk sync, the number of elements: 3 controller/src/dns/main.cc 120 2025-01-20 Mon 01:54:14:687.069 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139755028101248, Pid 106284]: DnsAgent [SYS_NOTICE]: DnsOperational: VirtualDnsConfig has been updated after the bulk sync, the number of elements: 0 controller/src/dns/main.cc 131 2025-01-20 Mon 01:54:14:687.077 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139755028101248, Pid 106284]: DnsAgent [SYS_NOTICE]: DnsOperational: VirtualDnsRecordConfig has been updated after the bulk sync, the number of elements: 0 controller/src/dns/main.cc 142 2025-01-20 Mon 01:54:14:687.085 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139755028101248, Pid 106284]: DnsAgent [SYS_NOTICE]: DnsOperational: GlobalQosConfig has been updated after the bulk sync controller/src/dns/main.cc 152 2025-01-20 Mon 01:54:14:687.132 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139755028101248, Pid 106284]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: RabbitMQ SM: RabbitMQ connection down src/contrail-common/config-client-mgr/config_amqp_client.cc 158 2025-01-20 Mon 01:54:14:687.142 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139755028101248, Pid 106284]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: RabbitMQ SM: Connect to Rabbit MQ with queue_delete TRUE src/contrail-common/config-client-mgr/config_amqp_client.cc 167 2025-01-20 Mon 01:54:14:687.690 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754844845824, Pid 106284]: Processing scm::EvSandeshSend in state Connect 2025-01-20 Mon 01:54:14:687.870 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754844845824, Pid 106284]: SANDESH: Send FAILED: 1737338054687115 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-2650-2 process_status= [ [ [ module_id = contrail-dns instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.42:8086, ] ] status = Initializing description = Disconnect : EvCollectorUpdate -> Connect ], [ type = Database name = Cassandra server_addrs= [ [ (*_iter6) = 10.0.0.42:9041, (*_iter6) = 10.0.0.22:9041, (*_iter6) = 10.0.0.49:9041, ] ] status = Up description = Established Cassandra connection ], [ type = Database name = RabbitMQ server_addrs= [ [ (*_iter6) = 10.0.0.42:5673, (*_iter6) = 10.0.0.22:5673, (*_iter6) = 10.0.0.49:5673, ] ] status = Down description = RabbitMQ connection down ], ] ] flag_infos= [ [ ] ] description = Collector, Database:RabbitMQ connection down ], ] ] ] 2025-01-20 Mon 01:54:14:687.885 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754844845824, Pid 106284]: Wrong state: Connect for event: EvSandeshSend message: NodeStatusUVE 2025-01-20 Mon 01:54:14:767.916 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139755028101248, Pid 106284]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: RabbitMQ SM: RabbitMQ connection established src/contrail-common/config-client-mgr/config_amqp_client.cc 151 2025-01-20 Mon 01:54:14:767.951 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754849044224, Pid 106284]: Processing scm::EvSandeshSend in state Connect 2025-01-20 Mon 01:54:14:768.136 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754849044224, Pid 106284]: SANDESH: Send FAILED: 1737338054767871 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-2650-2 process_status= [ [ [ module_id = contrail-dns instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.42:8086, ] ] status = Initializing description = Disconnect : EvCollectorUpdate -> Connect ], [ type = Database name = Cassandra server_addrs= [ [ (*_iter6) = 10.0.0.42:9041, (*_iter6) = 10.0.0.22:9041, (*_iter6) = 10.0.0.49:9041, ] ] status = Up description = Established Cassandra connection ], [ type = Database name = RabbitMQ server_addrs= [ [ (*_iter6) = 10.0.0.42:5673, (*_iter6) = 10.0.0.22:5673, (*_iter6) = 10.0.0.49:5673, ] ] status = Up description = RabbitMQ connection established ], ] ] flag_infos= [ [ ] ] description = Collector connection down ], ] ] ] 2025-01-20 Mon 01:54:14:768.149 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754849044224, Pid 106284]: Wrong state: Connect for event: EvSandeshSend message: NodeStatusUVE 2025-01-20 Mon 01:54:14:768.165 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754849044224, Pid 106284]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: TRUE src/contrail-common/config-client-mgr/config_client_manager.cc 225 2025-01-20 Mon 01:54:14:771.114 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139755028101248, Pid 106284]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.0.0.22:47726::10.0.0.42:8086(-1) < Active session connection complete src/contrail-common/io/tcp_session.cc 282 2025-01-20 Mon 01:54:14:771.140 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139755028101248, Pid 106284]: 10.0.0.22:47726::10.0.0.42:8086(-1) OnSessionEvent TCP Connected 2025-01-20 Mon 01:54:14:771.151 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139755028101248, Pid 106284]: 10.0.0.22:47726::10.0.0.42:8086(-1):EvTcpConnected 2025-01-20 Mon 01:54:14:772.268 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754853242624, Pid 106284]: DnsAgent [SYS_NOTICE]: DnsOperational: BIND named up; DNS is operational controller/src/dns/mgr/dns_mgr.cc 737 2025-01-20 Mon 01:54:14:772.307 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754853242624, Pid 106284]: Processing scm::EvTcpConnected in state Connect 2025-01-20 Mon 01:54:14:772.317 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754853242624, Pid 106284]: Connect : EvTcpConnected : Cancelling Connect timer 2025-01-20 Mon 01:54:14:772.353 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754853242624, Pid 106284]: ClientInit 2025-01-20 Mon 01:54:14:772.368 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754853242624, Pid 106284]: Sending Ctrl Message for cn-jenkins-deploy-platform-ansible-os-2650-2:contrail-dns:0:Control count 1 2025-01-20 Mon 01:54:14:772.474 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754853242624, Pid 106284]: Processing scm::EvSandeshSend in state ClientInit 2025-01-20 Mon 01:54:14:772.484 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754853242624, Pid 106284]: ClientInit : EvSandeshSend : NodeStatusUVE 2025-01-20 Mon 01:54:14:772.506 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754853242624, Pid 106284]: SANDESH: Send FAILED: 1737338054772345 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-2650-2 process_status= [ [ [ module_id = contrail-dns instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.42:8086, ] ] status = Initializing description = Connect : EvTcpConnected ], [ type = Database name = Cassandra server_addrs= [ [ (*_iter6) = 10.0.0.42:9041, (*_iter6) = 10.0.0.22:9041, (*_iter6) = 10.0.0.49:9041, ] ] status = Up description = Established Cassandra connection ], [ type = Database name = RabbitMQ server_addrs= [ [ (*_iter6) = 10.0.0.42:5673, (*_iter6) = 10.0.0.22:5673, (*_iter6) = 10.0.0.49:5673, ] ] status = Up description = RabbitMQ connection established ], ] ] flag_infos= [ [ ] ] description = Collector connection down ], ] ] ] 2025-01-20 Mon 01:54:14:772.513 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754853242624, Pid 106284]: Received UVE message in wrong state : NodeStatusUVE 2025-01-20 Mon 01:54:14:772.521 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754853242624, Pid 106284]: Processing scm::EvSandeshSend in state ClientInit 2025-01-20 Mon 01:54:14:772.527 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754853242624, Pid 106284]: ClientInit : EvSandeshSend : SandeshCtrlClientToServer 2025-01-20 Mon 01:54:14:772.539 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754853242624, Pid 106284]: Processing scm::EvSandeshSend in state ClientInit 2025-01-20 Mon 01:54:14:772.546 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754853242624, Pid 106284]: ClientInit : EvSandeshSend : SandeshModuleClientTrace 2025-01-20 Mon 01:54:14:772.696 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754853242624, Pid 106284]: SANDESH: Send FAILED: 1737338054772463 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-2650-2:Control:contrail-dns:0 client_info= [ status = ClientInit successful_connections = 1 pid = 106284 http_port = 8092 start_time = 1737338053624812 collector_name = collector_ip = 10.0.0.42:8086 collector_list= [ [ (*_iter40) = 10.0.0.42:8086, (*_iter40) = 10.0.0.49:8086, (*_iter40) = 10.0.0.22:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ConfigClientMgrDebug [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 152 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigurationMalformedReferenceWarningLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 14 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = DnsOperational [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 13 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapLinkOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 469 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapNodeOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 271 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 7 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 3 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 922, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 10, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-01-20 Mon 01:54:14:772.714 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754853242624, Pid 106284]: Received UVE message in wrong state : SandeshModuleClientTrace 2025-01-20 Mon 01:54:14:775.671 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754823853824, Pid 106284]: OnMessage control in state: ClientInit 2025-01-20 Mon 01:54:14:775.721 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754823853824, Pid 106284]: Processing scm::EvSandeshMessageRecv in state ClientInit 2025-01-20 Mon 01:54:14:775.735 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754823853824, Pid 106284]: ClientInit : EvSandeshMessageRecv 2025-01-20 Mon 01:54:14:775.767 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754823853824, Pid 106284]: Received Ctrl Message with size 0 2025-01-20 Mon 01:54:14:775.778 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754823853824, Pid 106284]: SyncAllMaps for DnsState:0 period 0 cycle 0 without seqno , total = 0 2025-01-20 Mon 01:54:14:779.946 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754823853824, Pid 106284]: SyncUVE Syncing ObjectGeneratorInfo val name = cn-jenkins-deploy-platform-ansible-os-2650-2:Control:contrail-dns:0 client_info= [ status = ClientInit successful_connections = 1 pid = 106284 http_port = 8092 start_time = 1737338053624812 collector_name = collector_ip = 10.0.0.42:8086 collector_list= [ [ (*_iter40) = 10.0.0.42:8086, (*_iter40) = 10.0.0.49:8086, (*_iter40) = 10.0.0.22:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ConfigClientMgrDebug [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 152 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigurationMalformedReferenceWarningLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 14 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = DnsOperational [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 13 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapLinkOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 469 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapNodeOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 271 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 7 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 3 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 922, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 10, _iter59->first = sent _iter59->second = 0, ] ] proxy seq 5 2025-01-20 Mon 01:54:14:779.997 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754823853824, Pid 106284]: SyncAllMaps for ModuleClientState:0 period 0 cycle 0 without seqno , total = 1 2025-01-20 Mon 01:54:14:780.023 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754823853824, Pid 106284]: SyncUVE Syncing ObjectDns val name = cn-jenkins-deploy-platform-ansible-os-2650-2 process_status= [ [ [ module_id = contrail-dns instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.42:8086, ] ] status = Initializing description = Connect : EvTcpConnected ], [ type = Database name = Cassandra server_addrs= [ [ (*_iter6) = 10.0.0.42:9041, (*_iter6) = 10.0.0.22:9041, (*_iter6) = 10.0.0.49:9041, ] ] status = Up description = Established Cassandra connection ], [ type = Database name = RabbitMQ server_addrs= [ [ (*_iter6) = 10.0.0.42:5673, (*_iter6) = 10.0.0.22:5673, (*_iter6) = 10.0.0.49:5673, ] ] status = Up description = RabbitMQ connection established ], ] ] flag_infos= [ [ ] ] description = Collector connection down ], ] ] proxy seq 8 2025-01-20 Mon 01:54:14:780.035 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754823853824, Pid 106284]: SyncAllMaps for NodeStatus:0 period 0 cycle 0 without seqno , total = 1 2025-01-20 Mon 01:54:14:780.046 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754823853824, Pid 106284]: SyncAllMaps for PeerFlapData:0 period 0 cycle 0 without seqno , total = 0 2025-01-20 Mon 01:54:14:780.052 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754823853824, Pid 106284]: SyncAllMaps for PeerStatsData:0 period 0 cycle 0 without seqno , total = 0 2025-01-20 Mon 01:54:14:780.060 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754823853824, Pid 106284]: SyncAllMaps for XmppPeerInfoData:0 period 0 cycle 0 without seqno , total = 0 2025-01-20 Mon 01:54:14:780.092 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754823853824, Pid 106284]: Established 2025-01-20 Mon 01:54:15:684.957 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754819655424, Pid 106284]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-01-20 Mon 01:54:15:838.515 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754819655424, Pid 106284]: Applying named configuration failed 2025-01-20 Mon 01:54:15:838.592 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754819655424, Pid 106284]: DnsAgent [SYS_NOTICE]: DnsOperational: Bulk sync has been done (the DB has been synced with named) controller/src/dns/mgr/dns_mgr.cc 455 2025-01-20 Mon 01:54:37:653.301 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139755028101248, Pid 106284]: XMPP [SYS_DEBUG]: XmppCreateConnection: XMPP Peer cn-jenkins-deploy-platform-ansible-os-2650-2:10.20.0.8 SEND Xmpp creating dynamic channel 10.20.0.11:8093::10.20.0.8:43025 controller/src/xmpp/xmpp_server.cc 586 2025-01-20 Mon 01:54:37:653.434 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139755028101248, Pid 106284]: XMPP [SYS_INFO]: XmppConnectionCreate: XMPP Peer cn-jenkins-deploy-platform-ansible-os-2650-2:10.20.0.8 RECV Created Xmpp Server connection from cn-jenkins-deploy-platform-ansible-os-2650-2 To : controller/src/xmpp/xmpp_connection.cc 727 2025-01-20 Mon 01:54:37:653.482 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139755028101248, Pid 106284]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.20.0.11:8093::10.20.0.8:43025 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 256 2025-01-20 Mon 01:54:37:653.588 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754870036224, Pid 106284]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer cn-jenkins-deploy-platform-ansible-os-2650-2:10.20.0.8 RECV Mode Server: PassiveOpen in state: Idle peer ip: 10.20.0.8 ( ) controller/src/xmpp/xmpp_state_machine.cc 1325 2025-01-20 Mon 01:54:37:653.663 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754840647424, Pid 106284]: [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.8 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-01-20 Mon 01:54:37:654.674 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754840647424, Pid 106284]: [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.8 state = Active event = EvTcpPassiveOpen local_port = 8093 remote_port = 43025 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-01-20 Mon 01:54:37:654.741 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754840647424, Pid 106284]: [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.8 state = Active next_state = Established event = EvXmppOpen identifier = an-jenkins-deploy-platform-ansible-os-2650-2/dns ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-01-20 Mon 01:54:38:146.129 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139755028101248, Pid 106284]: XMPP [SYS_DEBUG]: XmppCreateConnection: XMPP Peer cn-jenkins-deploy-platform-ansible-os-2650-2:10.20.0.13 SEND Xmpp creating dynamic channel 10.20.0.11:8093::10.20.0.13:32903 controller/src/xmpp/xmpp_server.cc 586 2025-01-20 Mon 01:54:38:146.197 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139755028101248, Pid 106284]: XMPP [SYS_INFO]: XmppConnectionCreate: XMPP Peer cn-jenkins-deploy-platform-ansible-os-2650-2:10.20.0.13 RECV Created Xmpp Server connection from cn-jenkins-deploy-platform-ansible-os-2650-2 To : controller/src/xmpp/xmpp_connection.cc 727 2025-01-20 Mon 01:54:38:146.238 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139755028101248, Pid 106284]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.20.0.11:8093::10.20.0.13:32903 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 256 2025-01-20 Mon 01:54:38:146.331 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754870036224, Pid 106284]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer cn-jenkins-deploy-platform-ansible-os-2650-2:10.20.0.13 RECV Mode Server: PassiveOpen in state: Idle peer ip: 10.20.0.13 ( ) controller/src/xmpp/xmpp_state_machine.cc 1325 2025-01-20 Mon 01:54:38:146.341 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754840647424, Pid 106284]: [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.13 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-01-20 Mon 01:54:38:146.649 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754840647424, Pid 106284]: [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.13 state = Active event = EvTcpPassiveOpen local_port = 8093 remote_port = 32903 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-01-20 Mon 01:54:38:147.482 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754853242624, Pid 106284]: [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.13 state = Active next_state = Established event = EvXmppOpen identifier = an-jenkins-deploy-platform-ansible-os-2650-1/dns ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-01-20 Mon 01:55:03:072.074 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754819655424, Pid 106284]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-01-20 Mon 01:55:31:986.268 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754849044224, Pid 106284]: ConfigClientManager [SYS_WARN]: ConfigClientMgrWarning: RabbitMQ SM: Caught exception while receiving messages from RabbitMQ: 10.0.0.42:5673 : connection error: 320: CONNECTION_FORCED - broker forced connection closure with reason 'shutdown' src/contrail-common/config-client-mgr/config_amqp_client.cc 366 2025-01-20 Mon 01:55:31:991.434 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754849044224, Pid 106284]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: RabbitMQ SM: RabbitMQ connection down src/contrail-common/config-client-mgr/config_amqp_client.cc 158 2025-01-20 Mon 01:55:31:991.466 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754849044224, Pid 106284]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: RabbitMQ SM: Connect to Rabbit MQ with queue_delete FALSE src/contrail-common/config-client-mgr/config_amqp_client.cc 167 2025-01-20 Mon 01:55:31:991.966 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754849044224, Pid 106284]: ConfigClientManager [SYS_WARN]: ConfigClientMgrWarning: RabbitMQ SM: Caught exception while connecting to RabbitMQ: 10.0.0.42:5673 : a socket error occurred src/contrail-common/config-client-mgr/config_amqp_client.cc 220 2025-01-20 Mon 01:55:32:035.209 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754849044224, Pid 106284]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: RabbitMQ SM: RabbitMQ connection established src/contrail-common/config-client-mgr/config_amqp_client.cc 151 2025-01-20 Mon 01:56:59:657.756 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139755028101248, Pid 106284]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.20.0.11:8092::10.20.0.18:44902 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 256 2025-01-20 Mon 01:57:00:946.676 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139755028101248, Pid 106284]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.20.0.11:8092::10.20.0.11:43812 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 256 2025-01-20 Mon 01:57:01:102.005 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139755028101248, Pid 106284]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.20.0.11:8092::10.20.0.7:53120 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 256 2025-01-20 Mon 01:57:11:140.320 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754849044224, Pid 106284]: ConfigClientManager [SYS_WARN]: ConfigClientMgrWarning: RabbitMQ SM: Caught exception while receiving messages from RabbitMQ: 10.0.0.22:5673 : connection error: 320: CONNECTION_FORCED - broker forced connection closure with reason 'shutdown' src/contrail-common/config-client-mgr/config_amqp_client.cc 366 2025-01-20 Mon 01:57:11:142.898 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754849044224, Pid 106284]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: RabbitMQ SM: RabbitMQ connection down src/contrail-common/config-client-mgr/config_amqp_client.cc 158 2025-01-20 Mon 01:57:11:142.925 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754849044224, Pid 106284]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: RabbitMQ SM: Connect to Rabbit MQ with queue_delete FALSE src/contrail-common/config-client-mgr/config_amqp_client.cc 167 2025-01-20 Mon 01:57:11:143.133 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754849044224, Pid 106284]: ConfigClientManager [SYS_WARN]: ConfigClientMgrWarning: RabbitMQ SM: Caught exception while connecting to RabbitMQ: 10.0.0.22:5673 : a socket error occurred src/contrail-common/config-client-mgr/config_amqp_client.cc 220 2025-01-20 Mon 01:57:11:492.881 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754849044224, Pid 106284]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: RabbitMQ SM: RabbitMQ connection established src/contrail-common/config-client-mgr/config_amqp_client.cc 151 2025-01-20 Mon 01:57:25:897.285 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139755028101248, Pid 106284]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.20.0.11:8092::10.20.0.18:47560 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 256 2025-01-20 Mon 01:57:26:028.911 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139755028101248, Pid 106284]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.20.0.11:8092::10.20.0.11:46040 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 256 2025-01-20 Mon 01:57:27:172.299 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139755028101248, Pid 106284]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.20.0.11:8092::10.20.0.7:40320 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 256 2025-01-20 Mon 01:58:45:493.809 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754849044224, Pid 106284]: ConfigClientManager [SYS_WARN]: ConfigClientMgrWarning: RabbitMQ SM: Caught exception while receiving messages from RabbitMQ: 10.0.0.49:5673 : connection error: 320: CONNECTION_FORCED - broker forced connection closure with reason 'shutdown' src/contrail-common/config-client-mgr/config_amqp_client.cc 366 2025-01-20 Mon 01:58:45:501.164 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754849044224, Pid 106284]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: RabbitMQ SM: RabbitMQ connection down src/contrail-common/config-client-mgr/config_amqp_client.cc 158 2025-01-20 Mon 01:58:45:501.194 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754849044224, Pid 106284]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: RabbitMQ SM: Connect to Rabbit MQ with queue_delete FALSE src/contrail-common/config-client-mgr/config_amqp_client.cc 167 2025-01-20 Mon 01:58:45:501.709 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754849044224, Pid 106284]: ConfigClientManager [SYS_WARN]: ConfigClientMgrWarning: RabbitMQ SM: Caught exception while connecting to RabbitMQ: 10.0.0.49:5673 : a socket error occurred src/contrail-common/config-client-mgr/config_amqp_client.cc 220 2025-01-20 Mon 01:58:45:856.444 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754849044224, Pid 106284]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: RabbitMQ SM: RabbitMQ connection established src/contrail-common/config-client-mgr/config_amqp_client.cc 151 2025-01-20 Mon 02:01:59:667.214 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139755028101248, Pid 106284]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.20.0.11:8092::10.20.0.18:58092 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 256 2025-01-20 Mon 02:02:00:951.891 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139755028101248, Pid 106284]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.20.0.11:8092::10.20.0.11:34794 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 256 2025-01-20 Mon 02:02:01:105.079 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139755028101248, Pid 106284]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.20.0.11:8092::10.20.0.7:33948 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 256 2025-01-20 Mon 02:02:25:900.370 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139755028101248, Pid 106284]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.20.0.11:8092::10.20.0.18:51870 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 256 2025-01-20 Mon 02:02:26:029.469 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139755028101248, Pid 106284]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.20.0.11:8092::10.20.0.11:41734 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 256 2025-01-20 Mon 02:02:27:170.603 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139755028101248, Pid 106284]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.20.0.11:8092::10.20.0.7:53294 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 256 2025-01-20 Mon 02:06:59:668.969 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139755028101248, Pid 106284]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.20.0.11:8092::10.20.0.18:57524 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 256 2025-01-20 Mon 02:07:00:956.008 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139755028101248, Pid 106284]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.20.0.11:8092::10.20.0.11:57992 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 256 2025-01-20 Mon 02:07:01:106.817 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139755028101248, Pid 106284]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.20.0.11:8092::10.20.0.7:47154 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 256 2025-01-20 Mon 02:07:25:901.729 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139755028101248, Pid 106284]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.20.0.11:8092::10.20.0.18:45494 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 256 2025-01-20 Mon 02:07:26:034.745 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139755028101248, Pid 106284]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.20.0.11:8092::10.20.0.11:41578 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 256 2025-01-20 Mon 02:07:27:179.088 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139755028101248, Pid 106284]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.20.0.11:8092::10.20.0.7:45258 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 256 2025-01-20 Mon 02:11:59:672.264 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139755028101248, Pid 106284]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.20.0.11:8092::10.20.0.18:54218 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 256 2025-01-20 Mon 02:12:00:962.498 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139755028101248, Pid 106284]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.20.0.11:8092::10.20.0.11:33208 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 256 2025-01-20 Mon 02:12:01:107.783 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139755028101248, Pid 106284]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.20.0.11:8092::10.20.0.7:36944 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 256 2025-01-20 Mon 02:12:25:904.335 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139755028101248, Pid 106284]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.20.0.11:8092::10.20.0.18:43234 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 256 2025-01-20 Mon 02:12:26:034.317 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139755028101248, Pid 106284]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.20.0.11:8092::10.20.0.11:35418 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 256 2025-01-20 Mon 02:12:27:178.004 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139755028101248, Pid 106284]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.20.0.11:8092::10.20.0.7:34278 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 256 2025-01-20 Mon 02:12:45:650.015 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139755028101248, Pid 106284]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer cn-jenkins-deploy-platform-ansible-os-2650-2:10.20.0.13 RECV Mode Server: Event: Tcp Connection Closed peer ip: 10.20.0.13 ( an-jenkins-deploy-platform-ansible-os-2650-1/dns ) controller/src/xmpp/xmpp_state_machine.cc 1310 2025-01-20 Mon 02:12:45:650.209 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754836449024, Pid 106284]: [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.13 state = Established next_state = Idle event = EvTcpClose ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-01-20 Mon 02:12:45:650.241 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754861639424, Pid 106284]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer cn-jenkins-deploy-platform-ansible-os-2650-2:10.20.0.13 Deleted Xmpp Server connection from cn-jenkins-deploy-platform-ansible-os-2650-2 To : an-jenkins-deploy-platform-ansible-os-2650-1/dns controller/src/xmpp/xmpp_connection.cc 734 2025-01-20 Mon 02:12:45:650.288 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754836449024, Pid 106284]: [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.13 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-01-20 Mon 02:12:45:737.753 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139755028101248, Pid 106284]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer cn-jenkins-deploy-platform-ansible-os-2650-2:10.20.0.8 RECV Mode Server: Event: Tcp Connection Closed peer ip: 10.20.0.8 ( an-jenkins-deploy-platform-ansible-os-2650-2/dns ) controller/src/xmpp/xmpp_state_machine.cc 1310 2025-01-20 Mon 02:12:45:737.906 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754840647424, Pid 106284]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer cn-jenkins-deploy-platform-ansible-os-2650-2:10.20.0.8 Deleted Xmpp Server connection from cn-jenkins-deploy-platform-ansible-os-2650-2 To : an-jenkins-deploy-platform-ansible-os-2650-2/dns controller/src/xmpp/xmpp_connection.cc 734 2025-01-20 Mon 02:12:45:737.941 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754819655424, Pid 106284]: [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.8 state = Established next_state = Idle event = EvTcpClose ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-01-20 Mon 02:12:45:738.015 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754819655424, Pid 106284]: [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.8 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-01-20 Mon 02:14:37:903.762 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139755028101248, Pid 106284]: XMPP [SYS_DEBUG]: XmppCreateConnection: XMPP Peer cn-jenkins-deploy-platform-ansible-os-2650-2:10.20.0.13 SEND Xmpp creating dynamic channel 10.20.0.11:8093::10.20.0.13:43007 controller/src/xmpp/xmpp_server.cc 586 2025-01-20 Mon 02:14:37:903.842 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139755028101248, Pid 106284]: XMPP [SYS_INFO]: XmppConnectionCreate: XMPP Peer cn-jenkins-deploy-platform-ansible-os-2650-2:10.20.0.13 RECV Created Xmpp Server connection from cn-jenkins-deploy-platform-ansible-os-2650-2 To : controller/src/xmpp/xmpp_connection.cc 727 2025-01-20 Mon 02:14:37:903.883 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139755028101248, Pid 106284]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.20.0.11:8093::10.20.0.13:43007 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 256 2025-01-20 Mon 02:14:37:903.980 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754853242624, Pid 106284]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer cn-jenkins-deploy-platform-ansible-os-2650-2:10.20.0.13 RECV Mode Server: PassiveOpen in state: Idle peer ip: 10.20.0.13 ( ) controller/src/xmpp/xmpp_state_machine.cc 1325 2025-01-20 Mon 02:14:37:904.168 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754828052224, Pid 106284]: [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.13 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-01-20 Mon 02:14:37:904.287 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754832250624, Pid 106284]: [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.13 state = Active event = EvTcpPassiveOpen local_port = 8093 remote_port = 43007 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-01-20 Mon 02:14:38:140.050 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754836449024, Pid 106284]: [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.13 state = Active next_state = Established event = EvXmppOpen identifier = an-jenkins-deploy-platform-ansible-os-2650-1/dns ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-01-20 Mon 02:14:39:263.821 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754819655424, Pid 106284]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-01-20 Mon 02:14:39:498.622 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754865837824, Pid 106284]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-01-20 Mon 02:15:29:368.570 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139755028101248, Pid 106284]: XMPP [SYS_DEBUG]: XmppCreateConnection: XMPP Peer cn-jenkins-deploy-platform-ansible-os-2650-2:10.20.0.8 SEND Xmpp creating dynamic channel 10.20.0.11:8093::10.20.0.8:38231 controller/src/xmpp/xmpp_server.cc 586 2025-01-20 Mon 02:15:29:368.638 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139755028101248, Pid 106284]: XMPP [SYS_INFO]: XmppConnectionCreate: XMPP Peer cn-jenkins-deploy-platform-ansible-os-2650-2:10.20.0.8 RECV Created Xmpp Server connection from cn-jenkins-deploy-platform-ansible-os-2650-2 To : controller/src/xmpp/xmpp_connection.cc 727 2025-01-20 Mon 02:15:29:368.676 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139755028101248, Pid 106284]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.20.0.11:8093::10.20.0.8:38231 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 256 2025-01-20 Mon 02:15:29:368.738 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754857441024, Pid 106284]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer cn-jenkins-deploy-platform-ansible-os-2650-2:10.20.0.8 RECV Mode Server: PassiveOpen in state: Idle peer ip: 10.20.0.8 ( ) controller/src/xmpp/xmpp_state_machine.cc 1325 2025-01-20 Mon 02:15:29:368.747 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754819655424, Pid 106284]: [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.8 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-01-20 Mon 02:15:29:369.206 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754840647424, Pid 106284]: [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.8 state = Active event = EvTcpPassiveOpen local_port = 8093 remote_port = 38231 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-01-20 Mon 02:15:29:372.310 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754828052224, Pid 106284]: [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.20.0.8 state = Active next_state = Established event = EvXmppOpen identifier = an-jenkins-deploy-platform-ansible-os-2650-2/dns ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-01-20 Mon 02:15:30:562.117 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754857441024, Pid 106284]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-01-20 Mon 02:15:30:808.376 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754840647424, Pid 106284]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-01-20 Mon 02:16:59:679.020 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139755028101248, Pid 106284]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.20.0.11:8092::10.20.0.18:38102 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 256 2025-01-20 Mon 02:17:00:963.995 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139755028101248, Pid 106284]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.20.0.11:8092::10.20.0.11:42364 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 256 2025-01-20 Mon 02:17:01:113.520 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139755028101248, Pid 106284]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.20.0.11:8092::10.20.0.7:49896 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 256 2025-01-20 Mon 02:17:25:910.844 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139755028101248, Pid 106284]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.20.0.11:8092::10.20.0.18:57562 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 256 2025-01-20 Mon 02:17:26:041.712 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139755028101248, Pid 106284]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.20.0.11:8092::10.20.0.11:39164 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 256 2025-01-20 Mon 02:17:27:183.062 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139755028101248, Pid 106284]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.20.0.11:8092::10.20.0.7:42108 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 256 2025-01-20 Mon 02:17:32:020.554 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754861639424, Pid 106284]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-01-20 Mon 02:17:32:020.750 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754836449024, Pid 106284]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating project:default-domain:admin controller/src/ifmap/ifmap_server_table.cc 95 2025-01-20 Mon 02:17:32:020.819 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754836449024, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating domain-project controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 02:17:32:095.880 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754840647424, Pid 106284]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-01-20 Mon 02:17:32:096.068 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754840647424, Pid 106284]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating application-policy-set:default-domain:admin:default-application-policy-set controller/src/ifmap/ifmap_server_table.cc 95 2025-01-20 Mon 02:17:32:096.137 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754840647424, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating project-application-policy-set controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 02:17:32:161.449 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754819655424, Pid 106284]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-01-20 Mon 02:17:32:161.637 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754819655424, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating project-default-application-policy-set controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 02:17:32:368.369 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754865837824, Pid 106284]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-01-20 Mon 02:17:32:368.578 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754865837824, Pid 106284]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating security-group:default-domain:admin:default controller/src/ifmap/ifmap_server_table.cc 95 2025-01-20 Mon 02:17:32:368.616 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754865837824, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating project-security-group controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 02:17:32:507.105 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754865837824, Pid 106284]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-01-20 Mon 02:17:32:563.856 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754853242624, Pid 106284]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-01-20 Mon 02:17:32:564.072 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754853242624, Pid 106284]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating access-control-list:default-domain:admin:default:ingress-access-control-list controller/src/ifmap/ifmap_server_table.cc 95 2025-01-20 Mon 02:17:32:564.122 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754853242624, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating security-group-access-control-list controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 02:17:32:689.312 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754861639424, Pid 106284]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-01-20 Mon 02:17:32:689.494 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754861639424, Pid 106284]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating access-control-list:default-domain:admin:default:egress-access-control-list controller/src/ifmap/ifmap_server_table.cc 95 2025-01-20 Mon 02:17:32:689.532 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754861639424, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating security-group-access-control-list controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 02:17:33:774.508 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139755028101248, Pid 106284]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.0.0.22:8092::10.0.0.42:35688 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 256 2025-01-20 Mon 02:17:36:079.002 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754823853824, Pid 106284]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-01-20 Mon 02:17:36:079.143 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754823853824, Pid 106284]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating project:default-domain:ctest-AnalyticsBasicTestSanity-78205111 controller/src/ifmap/ifmap_server_table.cc 95 2025-01-20 Mon 02:17:36:079.177 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754823853824, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating domain-project controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 02:17:36:158.877 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754840647424, Pid 106284]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-01-20 Mon 02:17:36:159.088 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754853242624, Pid 106284]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating application-policy-set:default-domain:ctest-AnalyticsBasicTestSanity-78205111:default-application-policy-set controller/src/ifmap/ifmap_server_table.cc 95 2025-01-20 Mon 02:17:36:159.147 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754853242624, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating project-application-policy-set controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 02:17:36:232.968 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754840647424, Pid 106284]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-01-20 Mon 02:17:36:233.170 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754840647424, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating project-default-application-policy-set controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 02:17:36:358.757 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754865837824, Pid 106284]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-01-20 Mon 02:17:36:358.946 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754865837824, Pid 106284]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating security-group:default-domain:ctest-AnalyticsBasicTestSanity-78205111:default controller/src/ifmap/ifmap_server_table.cc 95 2025-01-20 Mon 02:17:36:358.981 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754865837824, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating project-security-group controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 02:17:36:457.684 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754840647424, Pid 106284]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-01-20 Mon 02:17:36:457.857 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754840647424, Pid 106284]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating access-control-list:default-domain:ctest-AnalyticsBasicTestSanity-78205111:default:ingress-access-control-list controller/src/ifmap/ifmap_server_table.cc 95 2025-01-20 Mon 02:17:36:457.915 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754840647424, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating security-group-access-control-list controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 02:17:36:514.410 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754840647424, Pid 106284]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-01-20 Mon 02:17:36:578.143 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754865837824, Pid 106284]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-01-20 Mon 02:17:36:578.325 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754865837824, Pid 106284]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating access-control-list:default-domain:ctest-AnalyticsBasicTestSanity-78205111:default:egress-access-control-list controller/src/ifmap/ifmap_server_table.cc 95 2025-01-20 Mon 02:17:36:578.370 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754865837824, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating security-group-access-control-list controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 02:17:39:286.021 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754819655424, Pid 106284]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-01-20 Mon 02:17:39:806.642 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754857441024, Pid 106284]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-01-20 Mon 02:17:39:898.881 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754857441024, Pid 106284]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-01-20 Mon 02:17:39:899.048 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754857441024, Pid 106284]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating virtual-network:default-domain:ctest-AnalyticsBasicTestSanity-78205111:ctest-vn22-05159446 controller/src/ifmap/ifmap_server_table.cc 95 2025-01-20 Mon 02:17:39:899.097 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754857441024, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating project-virtual-network controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 02:17:39:968.900 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754857441024, Pid 106284]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-01-20 Mon 02:17:40:047.566 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754844845824, Pid 106284]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-01-20 Mon 02:17:40:111.370 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754853242624, Pid 106284]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-01-20 Mon 02:17:40:111.493 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754853242624, Pid 106284]: IFMap [SYS_WARN]: ConfigurationMalformedReferenceWarningLog: instance_target No entry in metadata map routing_instance b0f52465-8996-426f-a808-4e9798a142ea controller/src/ifmap/client/config_json_parser.cc 223 2025-01-20 Mon 02:17:40:300.125 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754853242624, Pid 106284]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-01-20 Mon 02:17:40:300.342 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754832250624, Pid 106284]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating virtual-network-network-ipam:attr(default-domain:ctest-AnalyticsBasicTestSanity-78205111:ctest-vn22-05159446,default-domain:default-project:default-network-ipam) controller/src/ifmap/ifmap_server_table.cc 95 2025-01-20 Mon 02:17:40:300.382 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754832250624, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating virtual-network-network-ipam controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 02:17:40:300.425 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754832250624, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating virtual-network-network-ipam controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 02:17:40:373.200 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754853242624, Pid 106284]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-01-20 Mon 02:17:41:079.725 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754853242624, Pid 106284]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-01-20 Mon 02:17:41:449.301 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754853242624, Pid 106284]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-01-20 Mon 02:17:45:501.336 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754865837824, Pid 106284]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-01-20 Mon 02:17:47:687.826 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754861639424, Pid 106284]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-01-20 Mon 02:17:47:763.857 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754828052224, Pid 106284]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-01-20 Mon 02:17:47:763.985 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754828052224, Pid 106284]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating virtual-machine:81087c6f-d654-49de-8bdd-eca4975409d4 controller/src/ifmap/ifmap_server_table.cc 95 2025-01-20 Mon 02:17:47:881.579 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754840647424, Pid 106284]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-01-20 Mon 02:17:47:881.772 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754840647424, Pid 106284]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating virtual-machine-interface:default-domain:ctest-AnalyticsBasicTestSanity-78205111:6e909866-2ac8-4511-b287-3088f7a7bdeb controller/src/ifmap/ifmap_server_table.cc 95 2025-01-20 Mon 02:17:47:881.812 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754840647424, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating project-virtual-machine-interface controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 02:17:47:881.839 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754840647424, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating virtual-machine-interface-security-group controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 02:17:47:881.863 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754840647424, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating virtual-machine-interface-virtual-machine controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 02:17:47:881.884 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754840647424, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating virtual-machine-interface-virtual-network controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 02:17:47:963.387 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754840647424, Pid 106284]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-01-20 Mon 02:17:48:103.763 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754853242624, Pid 106284]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-01-20 Mon 02:17:48:103.970 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754865837824, Pid 106284]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating instance-ip:12ff27bb-d12f-4ab2-be9e-72bbdc81a8d0 controller/src/ifmap/ifmap_server_table.cc 95 2025-01-20 Mon 02:17:48:104.032 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754865837824, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating instance-ip-virtual-machine-interface controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 02:17:48:104.065 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754865837824, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating instance-ip-virtual-network controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 02:17:48:860.057 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754865837824, Pid 106284]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-01-20 Mon 02:17:48:920.207 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754823853824, Pid 106284]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-01-20 Mon 02:18:03:115.114 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754844845824, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Deleting link controller/src/ifmap/ifmap_server_table.cc 145 2025-01-20 Mon 02:18:03:115.177 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754844845824, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Deleting link controller/src/ifmap/ifmap_server_table.cc 145 2025-01-20 Mon 02:18:03:115.195 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754844845824, Pid 106284]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Deleting instance-ip:12ff27bb-d12f-4ab2-be9e-72bbdc81a8d0 controller/src/ifmap/ifmap_server_table.cc 164 2025-01-20 Mon 02:18:03:115.223 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754844845824, Pid 106284]: IFMap [SYS_DEBUG]: LinkOper: LinkRemove instance-ip:12ff27bb-d12f-4ab2-be9e-72bbdc81a8d0 - virtual-machine-interface:default-domain:ctest-AnalyticsBasicTestSanity-78205111:6e909866-2ac8-4511-b287-3088f7a7bdeb , lhs: , rhs: controller/src/ifmap/ifmap_exporter.cc 574 2025-01-20 Mon 02:18:03:115.250 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754844845824, Pid 106284]: IFMap [SYS_DEBUG]: LinkOper: LinkRemove instance-ip:12ff27bb-d12f-4ab2-be9e-72bbdc81a8d0 - virtual-network:default-domain:ctest-AnalyticsBasicTestSanity-78205111:ctest-vn22-05159446 , lhs: , rhs: controller/src/ifmap/ifmap_exporter.cc 574 2025-01-20 Mon 02:18:03:198.665 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754819655424, Pid 106284]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-01-20 Mon 02:18:03:270.464 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754823853824, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Deleting link controller/src/ifmap/ifmap_server_table.cc 145 2025-01-20 Mon 02:18:03:270.529 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754823853824, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Deleting link controller/src/ifmap/ifmap_server_table.cc 145 2025-01-20 Mon 02:18:03:270.549 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754823853824, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Deleting link controller/src/ifmap/ifmap_server_table.cc 145 2025-01-20 Mon 02:18:03:270.567 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754823853824, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Deleting link controller/src/ifmap/ifmap_server_table.cc 145 2025-01-20 Mon 02:18:03:270.577 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754823853824, Pid 106284]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Deleting virtual-machine-interface:default-domain:ctest-AnalyticsBasicTestSanity-78205111:6e909866-2ac8-4511-b287-3088f7a7bdeb controller/src/ifmap/ifmap_server_table.cc 164 2025-01-20 Mon 02:18:03:270.598 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754823853824, Pid 106284]: IFMap [SYS_DEBUG]: LinkOper: LinkRemove virtual-machine-interface:default-domain:ctest-AnalyticsBasicTestSanity-78205111:6e909866-2ac8-4511-b287-3088f7a7bdeb - project:default-domain:ctest-AnalyticsBasicTestSanity-78205111 , lhs: , rhs: controller/src/ifmap/ifmap_exporter.cc 574 2025-01-20 Mon 02:18:03:270.623 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754823853824, Pid 106284]: IFMap [SYS_DEBUG]: LinkOper: LinkRemove virtual-machine-interface:default-domain:ctest-AnalyticsBasicTestSanity-78205111:6e909866-2ac8-4511-b287-3088f7a7bdeb - security-group:default-domain:ctest-AnalyticsBasicTestSanity-78205111:default , lhs: , rhs: controller/src/ifmap/ifmap_exporter.cc 574 2025-01-20 Mon 02:18:03:270.654 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754823853824, Pid 106284]: IFMap [SYS_DEBUG]: LinkOper: LinkRemove virtual-machine-interface:default-domain:ctest-AnalyticsBasicTestSanity-78205111:6e909866-2ac8-4511-b287-3088f7a7bdeb - virtual-machine:81087c6f-d654-49de-8bdd-eca4975409d4 , lhs: , rhs: controller/src/ifmap/ifmap_exporter.cc 574 2025-01-20 Mon 02:18:03:270.675 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754823853824, Pid 106284]: IFMap [SYS_DEBUG]: LinkOper: LinkRemove virtual-machine-interface:default-domain:ctest-AnalyticsBasicTestSanity-78205111:6e909866-2ac8-4511-b287-3088f7a7bdeb - virtual-network:default-domain:ctest-AnalyticsBasicTestSanity-78205111:ctest-vn22-05159446 , lhs: , rhs: controller/src/ifmap/ifmap_exporter.cc 574 2025-01-20 Mon 02:18:03:352.072 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754853242624, Pid 106284]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Deleting virtual-machine:81087c6f-d654-49de-8bdd-eca4975409d4 controller/src/ifmap/ifmap_server_table.cc 164 2025-01-20 Mon 02:18:04:534.340 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754853242624, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Deleting link controller/src/ifmap/ifmap_server_table.cc 145 2025-01-20 Mon 02:18:04:534.436 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754853242624, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Deleting link controller/src/ifmap/ifmap_server_table.cc 145 2025-01-20 Mon 02:18:04:534.459 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754853242624, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Deleting link controller/src/ifmap/ifmap_server_table.cc 145 2025-01-20 Mon 02:18:04:534.475 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754853242624, Pid 106284]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Deleting virtual-network:default-domain:ctest-AnalyticsBasicTestSanity-78205111:ctest-vn22-05159446 controller/src/ifmap/ifmap_server_table.cc 164 2025-01-20 Mon 02:18:04:534.489 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754853242624, Pid 106284]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Deleting virtual-network-network-ipam:attr(default-domain:ctest-AnalyticsBasicTestSanity-78205111:ctest-vn22-05159446,default-domain:default-project:default-network-ipam) controller/src/ifmap/ifmap_server_table.cc 164 2025-01-20 Mon 02:18:04:534.517 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754853242624, Pid 106284]: IFMap [SYS_DEBUG]: LinkOper: LinkRemove virtual-network:default-domain:ctest-AnalyticsBasicTestSanity-78205111:ctest-vn22-05159446 - project:default-domain:ctest-AnalyticsBasicTestSanity-78205111 , lhs: , rhs: controller/src/ifmap/ifmap_exporter.cc 574 2025-01-20 Mon 02:18:04:534.545 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754853242624, Pid 106284]: IFMap [SYS_DEBUG]: LinkOper: LinkRemove virtual-network:default-domain:ctest-AnalyticsBasicTestSanity-78205111:ctest-vn22-05159446 - virtual-network-network-ipam:attr(default-domain:ctest-AnalyticsBasicTestSanity-78205111:ctest-vn22-05159446,default-domain:default-project:default-network-ipam) , lhs: , rhs: controller/src/ifmap/ifmap_exporter.cc 574 2025-01-20 Mon 02:18:04:534.567 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754853242624, Pid 106284]: IFMap [SYS_DEBUG]: LinkOper: LinkRemove virtual-network-network-ipam:attr(default-domain:ctest-AnalyticsBasicTestSanity-78205111:ctest-vn22-05159446,default-domain:default-project:default-network-ipam) - network-ipam:default-domain:default-project:default-network-ipam , lhs: , rhs: controller/src/ifmap/ifmap_exporter.cc 574 2025-01-20 Mon 02:18:04:621.825 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754861639424, Pid 106284]: IFMap [SYS_WARN]: ConfigurationMalformedReferenceWarningLog: instance_target No entry in metadata map routing_instance b0f52465-8996-426f-a808-4e9798a142ea controller/src/ifmap/client/config_json_parser.cc 223 2025-01-20 Mon 02:18:04:884.612 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754857441024, Pid 106284]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-01-20 Mon 02:18:05:806.880 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754857441024, Pid 106284]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-01-20 Mon 02:18:05:888.290 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754823853824, Pid 106284]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-01-20 Mon 02:18:06:041.360 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754857441024, Pid 106284]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-01-20 Mon 02:18:06:121.754 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754844845824, Pid 106284]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-01-20 Mon 02:18:07:571.291 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754819655424, Pid 106284]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-01-20 Mon 02:18:07:571.444 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754819655424, Pid 106284]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating project:default-domain:ctest-FloatingipBasicTestSanity-53644544 controller/src/ifmap/ifmap_server_table.cc 95 2025-01-20 Mon 02:18:07:571.479 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754819655424, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating domain-project controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 02:18:07:678.676 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754819655424, Pid 106284]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-01-20 Mon 02:18:07:678.820 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754819655424, Pid 106284]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating application-policy-set:default-domain:ctest-FloatingipBasicTestSanity-53644544:default-application-policy-set controller/src/ifmap/ifmap_server_table.cc 95 2025-01-20 Mon 02:18:07:678.855 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754819655424, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating project-application-policy-set controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 02:18:07:757.402 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754844845824, Pid 106284]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-01-20 Mon 02:18:07:757.560 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754844845824, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating project-default-application-policy-set controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 02:18:07:929.928 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754819655424, Pid 106284]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-01-20 Mon 02:18:07:930.138 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754819655424, Pid 106284]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating security-group:default-domain:ctest-FloatingipBasicTestSanity-53644544:default controller/src/ifmap/ifmap_server_table.cc 95 2025-01-20 Mon 02:18:07:930.183 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754819655424, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating project-security-group controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 02:18:08:030.115 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754819655424, Pid 106284]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-01-20 Mon 02:18:08:030.295 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754819655424, Pid 106284]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating access-control-list:default-domain:ctest-FloatingipBasicTestSanity-53644544:default:ingress-access-control-list controller/src/ifmap/ifmap_server_table.cc 95 2025-01-20 Mon 02:18:08:030.383 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754819655424, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating security-group-access-control-list controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 02:18:08:086.884 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754819655424, Pid 106284]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-01-20 Mon 02:18:08:182.629 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754840647424, Pid 106284]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-01-20 Mon 02:18:08:182.846 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754840647424, Pid 106284]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating access-control-list:default-domain:ctest-FloatingipBasicTestSanity-53644544:default:egress-access-control-list controller/src/ifmap/ifmap_server_table.cc 95 2025-01-20 Mon 02:18:08:182.896 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754840647424, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating security-group-access-control-list controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 02:18:09:970.416 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754861639424, Pid 106284]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-01-20 Mon 02:18:10:968.982 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754861639424, Pid 106284]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-01-20 Mon 02:18:11:195.032 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754844845824, Pid 106284]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-01-20 Mon 02:18:11:258.223 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754844845824, Pid 106284]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-01-20 Mon 02:18:11:324.674 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754840647424, Pid 106284]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-01-20 Mon 02:18:12:453.843 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754844845824, Pid 106284]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-01-20 Mon 02:18:12:687.649 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754853242624, Pid 106284]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-01-20 Mon 02:18:12:802.978 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754844845824, Pid 106284]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-01-20 Mon 02:18:12:803.230 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754844845824, Pid 106284]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating virtual-network:default-domain:ctest-FloatingipBasicTestSanity-53644544:ctest-fvn-10107916 controller/src/ifmap/ifmap_server_table.cc 95 2025-01-20 Mon 02:18:12:803.302 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754844845824, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating project-virtual-network controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 02:18:12:907.122 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754853242624, Pid 106284]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-01-20 Mon 02:18:12:998.151 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754861639424, Pid 106284]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-01-20 Mon 02:18:13:048.046 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754857441024, Pid 106284]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-01-20 Mon 02:18:13:048.185 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754857441024, Pid 106284]: IFMap [SYS_WARN]: ConfigurationMalformedReferenceWarningLog: instance_target No entry in metadata map routing_instance bfbd0b3a-ac52-44db-a401-7c1c4b1421af controller/src/ifmap/client/config_json_parser.cc 223 2025-01-20 Mon 02:18:13:171.429 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754832250624, Pid 106284]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-01-20 Mon 02:18:13:171.698 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754857441024, Pid 106284]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating virtual-network-network-ipam:attr(default-domain:ctest-FloatingipBasicTestSanity-53644544:ctest-fvn-10107916,default-domain:default-project:default-network-ipam) controller/src/ifmap/ifmap_server_table.cc 95 2025-01-20 Mon 02:18:13:171.748 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754857441024, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating virtual-network-network-ipam controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 02:18:13:171.774 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754857441024, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating virtual-network-network-ipam controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 02:18:13:250.229 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754836449024, Pid 106284]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-01-20 Mon 02:18:13:951.691 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754861639424, Pid 106284]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-01-20 Mon 02:18:14:394.417 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754844845824, Pid 106284]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-01-20 Mon 02:18:15:761.994 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754865837824, Pid 106284]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-01-20 Mon 02:18:15:968.569 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754819655424, Pid 106284]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-01-20 Mon 02:18:16:070.533 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754832250624, Pid 106284]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-01-20 Mon 02:18:16:070.735 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754861639424, Pid 106284]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating virtual-network:default-domain:ctest-FloatingipBasicTestSanity-53644544:ctest-vn1-42957588 controller/src/ifmap/ifmap_server_table.cc 95 2025-01-20 Mon 02:18:16:070.822 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754861639424, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating project-virtual-network controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 02:18:16:161.919 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754836449024, Pid 106284]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-01-20 Mon 02:18:16:260.023 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754857441024, Pid 106284]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-01-20 Mon 02:18:16:309.284 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754853242624, Pid 106284]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-01-20 Mon 02:18:16:309.411 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754853242624, Pid 106284]: IFMap [SYS_WARN]: ConfigurationMalformedReferenceWarningLog: instance_target No entry in metadata map routing_instance 715fcbdf-3d49-4318-982f-866c88265060 controller/src/ifmap/client/config_json_parser.cc 223 2025-01-20 Mon 02:18:16:410.978 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754857441024, Pid 106284]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-01-20 Mon 02:18:16:411.214 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754836449024, Pid 106284]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating virtual-network-network-ipam:attr(default-domain:ctest-FloatingipBasicTestSanity-53644544:ctest-vn1-42957588,default-domain:default-project:default-network-ipam) controller/src/ifmap/ifmap_server_table.cc 95 2025-01-20 Mon 02:18:16:411.265 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754836449024, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating virtual-network-network-ipam controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 02:18:16:411.285 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754836449024, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating virtual-network-network-ipam controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 02:18:16:487.434 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754832250624, Pid 106284]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-01-20 Mon 02:18:16:827.969 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754823853824, Pid 106284]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-01-20 Mon 02:18:17:212.893 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754865837824, Pid 106284]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-01-20 Mon 02:18:18:500.554 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754844845824, Pid 106284]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-01-20 Mon 02:18:19:503.874 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754819655424, Pid 106284]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-01-20 Mon 02:18:19:572.471 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754832250624, Pid 106284]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-01-20 Mon 02:18:19:572.676 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754819655424, Pid 106284]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating virtual-machine:a586f188-0ea7-4e31-9913-ae1f524b3032 controller/src/ifmap/ifmap_server_table.cc 95 2025-01-20 Mon 02:18:19:717.077 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754853242624, Pid 106284]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-01-20 Mon 02:18:19:717.270 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754853242624, Pid 106284]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating virtual-machine-interface:default-domain:ctest-FloatingipBasicTestSanity-53644544:7ff2c356-09bc-457f-b2c1-b6fdd2da8c9e controller/src/ifmap/ifmap_server_table.cc 95 2025-01-20 Mon 02:18:19:717.309 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754853242624, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating project-virtual-machine-interface controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 02:18:19:717.337 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754853242624, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating virtual-machine-interface-security-group controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 02:18:19:717.356 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754853242624, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating virtual-machine-interface-virtual-machine controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 02:18:19:717.376 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754853242624, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating virtual-machine-interface-virtual-network controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 02:18:19:813.386 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754828052224, Pid 106284]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-01-20 Mon 02:18:19:966.161 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754853242624, Pid 106284]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-01-20 Mon 02:18:19:966.379 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754853242624, Pid 106284]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating instance-ip:5d4f741b-41a5-4e53-8fc4-a81daed858e3 controller/src/ifmap/ifmap_server_table.cc 95 2025-01-20 Mon 02:18:19:966.435 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754853242624, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating instance-ip-virtual-machine-interface controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 02:18:19:966.460 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754853242624, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating instance-ip-virtual-network controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 02:18:20:004.817 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754823853824, Pid 106284]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-01-20 Mon 02:18:20:296.578 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754828052224, Pid 106284]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-01-20 Mon 02:18:20:380.639 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754828052224, Pid 106284]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-01-20 Mon 02:18:20:688.936 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754828052224, Pid 106284]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-01-20 Mon 02:18:20:852.072 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754853242624, Pid 106284]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-01-20 Mon 02:18:20:852.246 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754853242624, Pid 106284]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating floating-ip-pool:default-domain:ctest-FloatingipBasicTestSanity-53644544:ctest-fvn-10107916:ctest-some-pool-61646292 controller/src/ifmap/ifmap_server_table.cc 95 2025-01-20 Mon 02:18:20:852.289 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754853242624, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating virtual-network-floating-ip-pool controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 02:18:20:926.634 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754836449024, Pid 106284]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-01-20 Mon 02:18:20:926.813 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754828052224, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating project-floating-ip-pool controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 02:18:21:148.213 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754823853824, Pid 106284]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-01-20 Mon 02:18:21:358.356 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754828052224, Pid 106284]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-01-20 Mon 02:18:21:892.236 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754823853824, Pid 106284]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-01-20 Mon 02:18:21:984.595 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754853242624, Pid 106284]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-01-20 Mon 02:18:21:984.737 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754853242624, Pid 106284]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating virtual-machine:fb948169-b1ec-4650-8b9a-f2dd4cac23c7 controller/src/ifmap/ifmap_server_table.cc 95 2025-01-20 Mon 02:18:22:158.769 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754836449024, Pid 106284]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-01-20 Mon 02:18:22:161.129 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754836449024, Pid 106284]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating virtual-machine-interface:default-domain:ctest-FloatingipBasicTestSanity-53644544:4ac5e18a-ac55-4e2f-aae1-038747c09aa0 controller/src/ifmap/ifmap_server_table.cc 95 2025-01-20 Mon 02:18:22:161.183 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754836449024, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating project-virtual-machine-interface controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 02:18:22:161.218 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754836449024, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating virtual-machine-interface-security-group controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 02:18:22:161.258 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754836449024, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating virtual-machine-interface-virtual-machine controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 02:18:22:161.279 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754836449024, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating virtual-machine-interface-virtual-network controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 02:18:22:223.742 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754823853824, Pid 106284]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-01-20 Mon 02:18:22:358.601 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754865837824, Pid 106284]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-01-20 Mon 02:18:22:358.760 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754865837824, Pid 106284]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating instance-ip:cac6dab8-590d-4e46-aaa8-d75b64aaca9b controller/src/ifmap/ifmap_server_table.cc 95 2025-01-20 Mon 02:18:22:358.800 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754865837824, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating instance-ip-virtual-machine-interface controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 02:18:22:358.828 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754865837824, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating instance-ip-virtual-network controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 02:18:23:031.911 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754865837824, Pid 106284]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-01-20 Mon 02:18:23:086.734 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754865837824, Pid 106284]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-01-20 Mon 02:18:38:063.642 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754840647424, Pid 106284]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-01-20 Mon 02:18:38:154.531 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754861639424, Pid 106284]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-01-20 Mon 02:18:38:154.732 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754853242624, Pid 106284]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating floating-ip:default-domain:ctest-FloatingipBasicTestSanity-53644544:ctest-fvn-10107916:ctest-some-pool-61646292:67c48aaf-dc27-4883-bd65-54fc13e103a9 controller/src/ifmap/ifmap_server_table.cc 95 2025-01-20 Mon 02:18:38:154.793 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754853242624, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating floating-ip-pool-floating-ip controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 02:18:38:154.838 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754853242624, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating floating-ip-project controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 02:18:38:329.948 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754823853824, Pid 106284]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-01-20 Mon 02:18:38:330.109 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754823853824, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating floating-ip-virtual-machine-interface controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 02:18:43:681.811 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754819655424, Pid 106284]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-01-20 Mon 02:18:43:682.015 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754819655424, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Deleting link controller/src/ifmap/ifmap_server_table.cc 145 2025-01-20 Mon 02:18:43:682.050 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754819655424, Pid 106284]: IFMap [SYS_DEBUG]: LinkOper: LinkRemove floating-ip:default-domain:ctest-FloatingipBasicTestSanity-53644544:ctest-fvn-10107916:ctest-some-pool-61646292:67c48aaf-dc27-4883-bd65-54fc13e103a9 - virtual-machine-interface:default-domain:ctest-FloatingipBasicTestSanity-53644544:7ff2c356-09bc-457f-b2c1-b6fdd2da8c9e , lhs: , rhs: controller/src/ifmap/ifmap_exporter.cc 574 2025-01-20 Mon 02:18:43:784.964 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754844845824, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Deleting link controller/src/ifmap/ifmap_server_table.cc 145 2025-01-20 Mon 02:18:43:785.012 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754844845824, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Deleting link controller/src/ifmap/ifmap_server_table.cc 145 2025-01-20 Mon 02:18:43:785.024 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754844845824, Pid 106284]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Deleting floating-ip:default-domain:ctest-FloatingipBasicTestSanity-53644544:ctest-fvn-10107916:ctest-some-pool-61646292:67c48aaf-dc27-4883-bd65-54fc13e103a9 controller/src/ifmap/ifmap_server_table.cc 164 2025-01-20 Mon 02:18:43:785.045 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754844845824, Pid 106284]: IFMap [SYS_DEBUG]: LinkOper: LinkRemove floating-ip:default-domain:ctest-FloatingipBasicTestSanity-53644544:ctest-fvn-10107916:ctest-some-pool-61646292:67c48aaf-dc27-4883-bd65-54fc13e103a9 - floating-ip-pool:default-domain:ctest-FloatingipBasicTestSanity-53644544:ctest-fvn-10107916:ctest-some-pool-61646292 , lhs: , rhs: controller/src/ifmap/ifmap_exporter.cc 574 2025-01-20 Mon 02:18:43:785.075 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754844845824, Pid 106284]: IFMap [SYS_DEBUG]: LinkOper: LinkRemove floating-ip:default-domain:ctest-FloatingipBasicTestSanity-53644544:ctest-fvn-10107916:ctest-some-pool-61646292:67c48aaf-dc27-4883-bd65-54fc13e103a9 - project:default-domain:ctest-FloatingipBasicTestSanity-53644544 , lhs: , rhs: controller/src/ifmap/ifmap_exporter.cc 574 2025-01-20 Mon 02:18:43:870.778 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754819655424, Pid 106284]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-01-20 Mon 02:18:43:870.943 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754819655424, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Deleting link controller/src/ifmap/ifmap_server_table.cc 145 2025-01-20 Mon 02:18:43:870.973 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754819655424, Pid 106284]: IFMap [SYS_DEBUG]: LinkOper: LinkRemove project:default-domain:ctest-FloatingipBasicTestSanity-53644544 - floating-ip-pool:default-domain:ctest-FloatingipBasicTestSanity-53644544:ctest-fvn-10107916:ctest-some-pool-61646292 , lhs: , rhs: controller/src/ifmap/ifmap_exporter.cc 574 2025-01-20 Mon 02:18:43:934.057 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754832250624, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Deleting link controller/src/ifmap/ifmap_server_table.cc 145 2025-01-20 Mon 02:18:43:934.096 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754832250624, Pid 106284]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Deleting floating-ip-pool:default-domain:ctest-FloatingipBasicTestSanity-53644544:ctest-fvn-10107916:ctest-some-pool-61646292 controller/src/ifmap/ifmap_server_table.cc 164 2025-01-20 Mon 02:18:43:934.117 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754832250624, Pid 106284]: IFMap [SYS_DEBUG]: LinkOper: LinkRemove floating-ip-pool:default-domain:ctest-FloatingipBasicTestSanity-53644544:ctest-fvn-10107916:ctest-some-pool-61646292 - virtual-network:default-domain:ctest-FloatingipBasicTestSanity-53644544:ctest-fvn-10107916 , lhs: , rhs: controller/src/ifmap/ifmap_exporter.cc 574 2025-01-20 Mon 02:18:44:769.959 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754857441024, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Deleting link controller/src/ifmap/ifmap_server_table.cc 145 2025-01-20 Mon 02:18:44:770.014 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754857441024, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Deleting link controller/src/ifmap/ifmap_server_table.cc 145 2025-01-20 Mon 02:18:44:770.027 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754857441024, Pid 106284]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Deleting instance-ip:cac6dab8-590d-4e46-aaa8-d75b64aaca9b controller/src/ifmap/ifmap_server_table.cc 164 2025-01-20 Mon 02:18:44:770.057 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754857441024, Pid 106284]: IFMap [SYS_DEBUG]: LinkOper: LinkRemove instance-ip:cac6dab8-590d-4e46-aaa8-d75b64aaca9b - virtual-machine-interface:default-domain:ctest-FloatingipBasicTestSanity-53644544:4ac5e18a-ac55-4e2f-aae1-038747c09aa0 , lhs: , rhs: controller/src/ifmap/ifmap_exporter.cc 574 2025-01-20 Mon 02:18:44:770.078 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754857441024, Pid 106284]: IFMap [SYS_DEBUG]: LinkOper: LinkRemove instance-ip:cac6dab8-590d-4e46-aaa8-d75b64aaca9b - virtual-network:default-domain:ctest-FloatingipBasicTestSanity-53644544:ctest-fvn-10107916 , lhs: , rhs: controller/src/ifmap/ifmap_exporter.cc 574 2025-01-20 Mon 02:18:44:903.725 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754836449024, Pid 106284]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-01-20 Mon 02:18:44:989.166 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754819655424, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Deleting link controller/src/ifmap/ifmap_server_table.cc 145 2025-01-20 Mon 02:18:44:989.215 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754819655424, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Deleting link controller/src/ifmap/ifmap_server_table.cc 145 2025-01-20 Mon 02:18:44:989.227 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754819655424, Pid 106284]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Deleting instance-ip:5d4f741b-41a5-4e53-8fc4-a81daed858e3 controller/src/ifmap/ifmap_server_table.cc 164 2025-01-20 Mon 02:18:44:989.249 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754819655424, Pid 106284]: IFMap [SYS_DEBUG]: LinkOper: LinkRemove instance-ip:5d4f741b-41a5-4e53-8fc4-a81daed858e3 - virtual-machine-interface:default-domain:ctest-FloatingipBasicTestSanity-53644544:7ff2c356-09bc-457f-b2c1-b6fdd2da8c9e , lhs: , rhs: controller/src/ifmap/ifmap_exporter.cc 574 2025-01-20 Mon 02:18:44:989.270 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754819655424, Pid 106284]: IFMap [SYS_DEBUG]: LinkOper: LinkRemove instance-ip:5d4f741b-41a5-4e53-8fc4-a81daed858e3 - virtual-network:default-domain:ctest-FloatingipBasicTestSanity-53644544:ctest-vn1-42957588 , lhs: , rhs: controller/src/ifmap/ifmap_exporter.cc 574 2025-01-20 Mon 02:18:45:028.277 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754836449024, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Deleting link controller/src/ifmap/ifmap_server_table.cc 145 2025-01-20 Mon 02:18:45:028.336 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754836449024, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Deleting link controller/src/ifmap/ifmap_server_table.cc 145 2025-01-20 Mon 02:18:45:028.356 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754836449024, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Deleting link controller/src/ifmap/ifmap_server_table.cc 145 2025-01-20 Mon 02:18:45:028.375 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754836449024, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Deleting link controller/src/ifmap/ifmap_server_table.cc 145 2025-01-20 Mon 02:18:45:028.408 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754836449024, Pid 106284]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Deleting virtual-machine-interface:default-domain:ctest-FloatingipBasicTestSanity-53644544:4ac5e18a-ac55-4e2f-aae1-038747c09aa0 controller/src/ifmap/ifmap_server_table.cc 164 2025-01-20 Mon 02:18:45:028.430 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754836449024, Pid 106284]: IFMap [SYS_DEBUG]: LinkOper: LinkRemove virtual-machine-interface:default-domain:ctest-FloatingipBasicTestSanity-53644544:4ac5e18a-ac55-4e2f-aae1-038747c09aa0 - project:default-domain:ctest-FloatingipBasicTestSanity-53644544 , lhs: , rhs: controller/src/ifmap/ifmap_exporter.cc 574 2025-01-20 Mon 02:18:45:028.452 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754836449024, Pid 106284]: IFMap [SYS_DEBUG]: LinkOper: LinkRemove virtual-machine-interface:default-domain:ctest-FloatingipBasicTestSanity-53644544:4ac5e18a-ac55-4e2f-aae1-038747c09aa0 - security-group:default-domain:ctest-FloatingipBasicTestSanity-53644544:default , lhs: , rhs: controller/src/ifmap/ifmap_exporter.cc 574 2025-01-20 Mon 02:18:45:028.466 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754836449024, Pid 106284]: IFMap [SYS_DEBUG]: LinkOper: LinkRemove virtual-machine-interface:default-domain:ctest-FloatingipBasicTestSanity-53644544:4ac5e18a-ac55-4e2f-aae1-038747c09aa0 - virtual-machine:fb948169-b1ec-4650-8b9a-f2dd4cac23c7 , lhs: , rhs: controller/src/ifmap/ifmap_exporter.cc 574 2025-01-20 Mon 02:18:45:028.480 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754836449024, Pid 106284]: IFMap [SYS_DEBUG]: LinkOper: LinkRemove virtual-machine-interface:default-domain:ctest-FloatingipBasicTestSanity-53644544:4ac5e18a-ac55-4e2f-aae1-038747c09aa0 - virtual-network:default-domain:ctest-FloatingipBasicTestSanity-53644544:ctest-fvn-10107916 , lhs: , rhs: controller/src/ifmap/ifmap_exporter.cc 574 2025-01-20 Mon 02:18:45:094.181 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754857441024, Pid 106284]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Deleting virtual-machine:fb948169-b1ec-4650-8b9a-f2dd4cac23c7 controller/src/ifmap/ifmap_server_table.cc 164 2025-01-20 Mon 02:18:45:134.885 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754853242624, Pid 106284]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-01-20 Mon 02:18:45:232.048 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754836449024, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Deleting link controller/src/ifmap/ifmap_server_table.cc 145 2025-01-20 Mon 02:18:45:232.102 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754836449024, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Deleting link controller/src/ifmap/ifmap_server_table.cc 145 2025-01-20 Mon 02:18:45:232.122 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754836449024, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Deleting link controller/src/ifmap/ifmap_server_table.cc 145 2025-01-20 Mon 02:18:45:232.150 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754836449024, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Deleting link controller/src/ifmap/ifmap_server_table.cc 145 2025-01-20 Mon 02:18:45:232.161 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754836449024, Pid 106284]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Deleting virtual-machine-interface:default-domain:ctest-FloatingipBasicTestSanity-53644544:7ff2c356-09bc-457f-b2c1-b6fdd2da8c9e controller/src/ifmap/ifmap_server_table.cc 164 2025-01-20 Mon 02:18:45:232.183 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754836449024, Pid 106284]: IFMap [SYS_DEBUG]: LinkOper: LinkRemove virtual-machine-interface:default-domain:ctest-FloatingipBasicTestSanity-53644544:7ff2c356-09bc-457f-b2c1-b6fdd2da8c9e - project:default-domain:ctest-FloatingipBasicTestSanity-53644544 , lhs: , rhs: controller/src/ifmap/ifmap_exporter.cc 574 2025-01-20 Mon 02:18:45:232.204 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754836449024, Pid 106284]: IFMap [SYS_DEBUG]: LinkOper: LinkRemove virtual-machine-interface:default-domain:ctest-FloatingipBasicTestSanity-53644544:7ff2c356-09bc-457f-b2c1-b6fdd2da8c9e - security-group:default-domain:ctest-FloatingipBasicTestSanity-53644544:default , lhs: , rhs: controller/src/ifmap/ifmap_exporter.cc 574 2025-01-20 Mon 02:18:45:232.220 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754836449024, Pid 106284]: IFMap [SYS_DEBUG]: LinkOper: LinkRemove virtual-machine-interface:default-domain:ctest-FloatingipBasicTestSanity-53644544:7ff2c356-09bc-457f-b2c1-b6fdd2da8c9e - virtual-machine:a586f188-0ea7-4e31-9913-ae1f524b3032 , lhs: , rhs: controller/src/ifmap/ifmap_exporter.cc 574 2025-01-20 Mon 02:18:45:232.234 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754836449024, Pid 106284]: IFMap [SYS_DEBUG]: LinkOper: LinkRemove virtual-machine-interface:default-domain:ctest-FloatingipBasicTestSanity-53644544:7ff2c356-09bc-457f-b2c1-b6fdd2da8c9e - virtual-network:default-domain:ctest-FloatingipBasicTestSanity-53644544:ctest-vn1-42957588 , lhs: , rhs: controller/src/ifmap/ifmap_exporter.cc 574 2025-01-20 Mon 02:18:45:287.093 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754861639424, Pid 106284]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Deleting virtual-machine:a586f188-0ea7-4e31-9913-ae1f524b3032 controller/src/ifmap/ifmap_server_table.cc 164 2025-01-20 Mon 02:18:46:349.680 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754836449024, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Deleting link controller/src/ifmap/ifmap_server_table.cc 145 2025-01-20 Mon 02:18:46:349.754 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754836449024, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Deleting link controller/src/ifmap/ifmap_server_table.cc 145 2025-01-20 Mon 02:18:46:349.775 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754836449024, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Deleting link controller/src/ifmap/ifmap_server_table.cc 145 2025-01-20 Mon 02:18:46:349.790 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754836449024, Pid 106284]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Deleting virtual-network:default-domain:ctest-FloatingipBasicTestSanity-53644544:ctest-vn1-42957588 controller/src/ifmap/ifmap_server_table.cc 164 2025-01-20 Mon 02:18:46:349.817 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754836449024, Pid 106284]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Deleting virtual-network-network-ipam:attr(default-domain:ctest-FloatingipBasicTestSanity-53644544:ctest-vn1-42957588,default-domain:default-project:default-network-ipam) controller/src/ifmap/ifmap_server_table.cc 164 2025-01-20 Mon 02:18:46:349.845 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754836449024, Pid 106284]: IFMap [SYS_DEBUG]: LinkOper: LinkRemove virtual-network:default-domain:ctest-FloatingipBasicTestSanity-53644544:ctest-vn1-42957588 - project:default-domain:ctest-FloatingipBasicTestSanity-53644544 , lhs: , rhs: controller/src/ifmap/ifmap_exporter.cc 574 2025-01-20 Mon 02:18:46:349.874 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754836449024, Pid 106284]: IFMap [SYS_DEBUG]: LinkOper: LinkRemove virtual-network:default-domain:ctest-FloatingipBasicTestSanity-53644544:ctest-vn1-42957588 - virtual-network-network-ipam:attr(default-domain:ctest-FloatingipBasicTestSanity-53644544:ctest-vn1-42957588,default-domain:default-project:default-network-ipam) , lhs: , rhs: controller/src/ifmap/ifmap_exporter.cc 574 2025-01-20 Mon 02:18:46:349.895 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754836449024, Pid 106284]: IFMap [SYS_DEBUG]: LinkOper: LinkRemove virtual-network-network-ipam:attr(default-domain:ctest-FloatingipBasicTestSanity-53644544:ctest-vn1-42957588,default-domain:default-project:default-network-ipam) - network-ipam:default-domain:default-project:default-network-ipam , lhs: , rhs: controller/src/ifmap/ifmap_exporter.cc 574 2025-01-20 Mon 02:18:46:428.465 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754832250624, Pid 106284]: IFMap [SYS_WARN]: ConfigurationMalformedReferenceWarningLog: instance_target No entry in metadata map routing_instance 715fcbdf-3d49-4318-982f-866c88265060 controller/src/ifmap/client/config_json_parser.cc 223 2025-01-20 Mon 02:18:46:655.345 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754870036224, Pid 106284]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-01-20 Mon 02:18:46:874.711 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754836449024, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Deleting link controller/src/ifmap/ifmap_server_table.cc 145 2025-01-20 Mon 02:18:46:874.762 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754836449024, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Deleting link controller/src/ifmap/ifmap_server_table.cc 145 2025-01-20 Mon 02:18:46:874.778 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754836449024, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Deleting link controller/src/ifmap/ifmap_server_table.cc 145 2025-01-20 Mon 02:18:46:874.789 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754836449024, Pid 106284]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Deleting virtual-network:default-domain:ctest-FloatingipBasicTestSanity-53644544:ctest-fvn-10107916 controller/src/ifmap/ifmap_server_table.cc 164 2025-01-20 Mon 02:18:46:874.809 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754836449024, Pid 106284]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Deleting virtual-network-network-ipam:attr(default-domain:ctest-FloatingipBasicTestSanity-53644544:ctest-fvn-10107916,default-domain:default-project:default-network-ipam) controller/src/ifmap/ifmap_server_table.cc 164 2025-01-20 Mon 02:18:46:874.830 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754836449024, Pid 106284]: IFMap [SYS_DEBUG]: LinkOper: LinkRemove virtual-network:default-domain:ctest-FloatingipBasicTestSanity-53644544:ctest-fvn-10107916 - project:default-domain:ctest-FloatingipBasicTestSanity-53644544 , lhs: , rhs: controller/src/ifmap/ifmap_exporter.cc 574 2025-01-20 Mon 02:18:46:874.850 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754836449024, Pid 106284]: IFMap [SYS_DEBUG]: LinkOper: LinkRemove virtual-network:default-domain:ctest-FloatingipBasicTestSanity-53644544:ctest-fvn-10107916 - virtual-network-network-ipam:attr(default-domain:ctest-FloatingipBasicTestSanity-53644544:ctest-fvn-10107916,default-domain:default-project:default-network-ipam) , lhs: , rhs: controller/src/ifmap/ifmap_exporter.cc 574 2025-01-20 Mon 02:18:46:874.865 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754836449024, Pid 106284]: IFMap [SYS_DEBUG]: LinkOper: LinkRemove virtual-network-network-ipam:attr(default-domain:ctest-FloatingipBasicTestSanity-53644544:ctest-fvn-10107916,default-domain:default-project:default-network-ipam) - network-ipam:default-domain:default-project:default-network-ipam , lhs: , rhs: controller/src/ifmap/ifmap_exporter.cc 574 2025-01-20 Mon 02:18:46:952.226 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754836449024, Pid 106284]: IFMap [SYS_WARN]: ConfigurationMalformedReferenceWarningLog: instance_target No entry in metadata map routing_instance bfbd0b3a-ac52-44db-a401-7c1c4b1421af controller/src/ifmap/client/config_json_parser.cc 223 2025-01-20 Mon 02:18:47:151.421 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754836449024, Pid 106284]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-01-20 Mon 02:18:48:620.812 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754832250624, Pid 106284]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-01-20 Mon 02:18:48:684.831 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754861639424, Pid 106284]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-01-20 Mon 02:18:48:821.527 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754853242624, Pid 106284]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-01-20 Mon 02:18:48:937.212 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754823853824, Pid 106284]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-01-20 Mon 02:18:50:163.767 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754853242624, Pid 106284]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-01-20 Mon 02:18:50:163.931 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754861639424, Pid 106284]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating project:default-domain:ctest-TestHeat-71270079 controller/src/ifmap/ifmap_server_table.cc 95 2025-01-20 Mon 02:18:50:163.999 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754861639424, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating domain-project controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 02:18:50:222.306 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754844845824, Pid 106284]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-01-20 Mon 02:18:50:222.456 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754844845824, Pid 106284]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating application-policy-set:default-domain:ctest-TestHeat-71270079:default-application-policy-set controller/src/ifmap/ifmap_server_table.cc 95 2025-01-20 Mon 02:18:50:222.494 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754844845824, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating project-application-policy-set controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 02:18:50:281.011 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754861639424, Pid 106284]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-01-20 Mon 02:18:50:281.177 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754861639424, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating project-default-application-policy-set controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 02:18:50:465.767 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754853242624, Pid 106284]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-01-20 Mon 02:18:50:465.967 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754853242624, Pid 106284]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating security-group:default-domain:ctest-TestHeat-71270079:default controller/src/ifmap/ifmap_server_table.cc 95 2025-01-20 Mon 02:18:50:466.003 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754853242624, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating project-security-group controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 02:18:50:545.957 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754853242624, Pid 106284]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-01-20 Mon 02:18:50:546.123 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754853242624, Pid 106284]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating access-control-list:default-domain:ctest-TestHeat-71270079:default:ingress-access-control-list controller/src/ifmap/ifmap_server_table.cc 95 2025-01-20 Mon 02:18:50:546.162 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754853242624, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating security-group-access-control-list controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 02:18:50:620.070 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754853242624, Pid 106284]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-01-20 Mon 02:18:50:682.297 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754853242624, Pid 106284]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-01-20 Mon 02:18:50:682.480 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754853242624, Pid 106284]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating access-control-list:default-domain:ctest-TestHeat-71270079:default:egress-access-control-list controller/src/ifmap/ifmap_server_table.cc 95 2025-01-20 Mon 02:18:50:682.540 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754853242624, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating security-group-access-control-list controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 02:18:55:210.612 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754857441024, Pid 106284]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-01-20 Mon 02:18:55:299.387 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754865837824, Pid 106284]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-01-20 Mon 02:18:55:415.145 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754836449024, Pid 106284]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-01-20 Mon 02:18:55:502.979 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754828052224, Pid 106284]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-01-20 Mon 02:18:56:523.606 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754828052224, Pid 106284]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-01-20 Mon 02:18:56:538.790 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754870036224, Pid 106284]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-01-20 Mon 02:18:56:560.709 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754836449024, Pid 106284]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-01-20 Mon 02:18:56:694.026 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754865837824, Pid 106284]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-01-20 Mon 02:18:56:727.612 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754857441024, Pid 106284]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-01-20 Mon 02:18:56:771.533 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754832250624, Pid 106284]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-01-20 Mon 02:18:56:875.526 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754853242624, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Deleting link controller/src/ifmap/ifmap_server_table.cc 145 2025-01-20 Mon 02:18:56:875.622 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754853242624, Pid 106284]: IFMap [SYS_DEBUG]: LinkOper: LinkRemove security-group:default-domain:ctest-AnalyticsBasicTestSanity-78205111:default - project:default-domain:ctest-AnalyticsBasicTestSanity-78205111 , lhs: , rhs: controller/src/ifmap/ifmap_exporter.cc 574 2025-01-20 Mon 02:18:56:893.688 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754870036224, Pid 106284]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-01-20 Mon 02:18:56:899.565 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754823853824, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Deleting link controller/src/ifmap/ifmap_server_table.cc 145 2025-01-20 Mon 02:18:56:899.624 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754823853824, Pid 106284]: IFMap [SYS_DEBUG]: LinkOper: LinkRemove security-group:default-domain:ctest-TestHeat-71270079:default - project:default-domain:ctest-TestHeat-71270079 , lhs: , rhs: controller/src/ifmap/ifmap_exporter.cc 574 2025-01-20 Mon 02:18:56:975.003 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754870036224, Pid 106284]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-01-20 Mon 02:18:57:001.513 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754819655424, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Deleting link controller/src/ifmap/ifmap_server_table.cc 145 2025-01-20 Mon 02:18:57:001.565 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754819655424, Pid 106284]: IFMap [SYS_DEBUG]: LinkOper: LinkRemove security-group:default-domain:ctest-FloatingipBasicTestSanity-53644544:default - project:default-domain:ctest-FloatingipBasicTestSanity-53644544 , lhs: , rhs: controller/src/ifmap/ifmap_exporter.cc 574 2025-01-20 Mon 02:18:57:070.899 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754861639424, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Deleting link controller/src/ifmap/ifmap_server_table.cc 145 2025-01-20 Mon 02:18:57:070.941 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754861639424, Pid 106284]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Deleting access-control-list:default-domain:ctest-AnalyticsBasicTestSanity-78205111:default:ingress-access-control-list controller/src/ifmap/ifmap_server_table.cc 164 2025-01-20 Mon 02:18:57:070.966 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754861639424, Pid 106284]: IFMap [SYS_DEBUG]: LinkOper: LinkRemove access-control-list:default-domain:ctest-AnalyticsBasicTestSanity-78205111:default:ingress-access-control-list - security-group:default-domain:ctest-AnalyticsBasicTestSanity-78205111:default , lhs: , rhs: controller/src/ifmap/ifmap_exporter.cc 574 2025-01-20 Mon 02:18:57:115.943 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754857441024, Pid 106284]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-01-20 Mon 02:18:57:154.195 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754857441024, Pid 106284]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-01-20 Mon 02:18:57:202.260 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754844845824, Pid 106284]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-01-20 Mon 02:18:57:267.092 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754870036224, Pid 106284]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-01-20 Mon 02:18:57:302.131 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754840647424, Pid 106284]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-01-20 Mon 02:18:57:340.060 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754836449024, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Deleting link controller/src/ifmap/ifmap_server_table.cc 145 2025-01-20 Mon 02:18:57:340.098 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754836449024, Pid 106284]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Deleting access-control-list:default-domain:ctest-AnalyticsBasicTestSanity-78205111:default:egress-access-control-list controller/src/ifmap/ifmap_server_table.cc 164 2025-01-20 Mon 02:18:57:340.109 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754836449024, Pid 106284]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Deleting security-group:default-domain:ctest-AnalyticsBasicTestSanity-78205111:default controller/src/ifmap/ifmap_server_table.cc 164 2025-01-20 Mon 02:18:57:340.131 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754836449024, Pid 106284]: IFMap [SYS_DEBUG]: LinkOper: LinkRemove access-control-list:default-domain:ctest-AnalyticsBasicTestSanity-78205111:default:egress-access-control-list - security-group:default-domain:ctest-AnalyticsBasicTestSanity-78205111:default , lhs: , rhs: controller/src/ifmap/ifmap_exporter.cc 574 2025-01-20 Mon 02:18:57:392.417 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754853242624, Pid 106284]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-01-20 Mon 02:18:57:406.308 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754844845824, Pid 106284]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-01-20 Mon 02:18:57:431.302 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754840647424, Pid 106284]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-01-20 Mon 02:18:57:431.485 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754865837824, Pid 106284]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating project:default-domain:ctest-TestRoutersBasic-14108397 controller/src/ifmap/ifmap_server_table.cc 95 2025-01-20 Mon 02:18:57:431.538 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754865837824, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating domain-project controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 02:18:57:468.238 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754857441024, Pid 106284]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-01-20 Mon 02:18:57:468.408 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754857441024, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Deleting link controller/src/ifmap/ifmap_server_table.cc 145 2025-01-20 Mon 02:18:57:468.439 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754857441024, Pid 106284]: IFMap [SYS_DEBUG]: LinkOper: LinkRemove project:default-domain:ctest-AnalyticsBasicTestSanity-78205111 - application-policy-set:default-domain:ctest-AnalyticsBasicTestSanity-78205111:default-application-policy-set , lhs: , rhs: controller/src/ifmap/ifmap_exporter.cc 574 2025-01-20 Mon 02:18:57:502.143 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754840647424, Pid 106284]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-01-20 Mon 02:18:57:502.317 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754840647424, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Deleting link controller/src/ifmap/ifmap_server_table.cc 145 2025-01-20 Mon 02:18:57:502.351 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754840647424, Pid 106284]: IFMap [SYS_DEBUG]: LinkOper: LinkRemove project:default-domain:ctest-TestHeat-71270079 - application-policy-set:default-domain:ctest-TestHeat-71270079:default-application-policy-set , lhs: , rhs: controller/src/ifmap/ifmap_exporter.cc 574 2025-01-20 Mon 02:18:57:533.456 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754840647424, Pid 106284]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-01-20 Mon 02:18:57:559.166 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754870036224, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Deleting link controller/src/ifmap/ifmap_server_table.cc 145 2025-01-20 Mon 02:18:57:559.209 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754870036224, Pid 106284]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Deleting access-control-list:default-domain:ctest-TestHeat-71270079:default:ingress-access-control-list controller/src/ifmap/ifmap_server_table.cc 164 2025-01-20 Mon 02:18:57:559.234 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754870036224, Pid 106284]: IFMap [SYS_DEBUG]: LinkOper: LinkRemove access-control-list:default-domain:ctest-TestHeat-71270079:default:ingress-access-control-list - security-group:default-domain:ctest-TestHeat-71270079:default , lhs: , rhs: controller/src/ifmap/ifmap_exporter.cc 574 2025-01-20 Mon 02:18:57:591.683 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754828052224, Pid 106284]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-01-20 Mon 02:18:57:591.835 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754828052224, Pid 106284]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating application-policy-set:default-domain:ctest-TestRoutersBasic-14108397:default-application-policy-set controller/src/ifmap/ifmap_server_table.cc 95 2025-01-20 Mon 02:18:57:591.879 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754828052224, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating project-application-policy-set controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 02:18:57:597.628 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754832250624, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Deleting link controller/src/ifmap/ifmap_server_table.cc 145 2025-01-20 Mon 02:18:57:597.670 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754832250624, Pid 106284]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Deleting application-policy-set:default-domain:ctest-AnalyticsBasicTestSanity-78205111:default-application-policy-set controller/src/ifmap/ifmap_server_table.cc 164 2025-01-20 Mon 02:18:57:597.694 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754832250624, Pid 106284]: IFMap [SYS_DEBUG]: LinkOper: LinkRemove application-policy-set:default-domain:ctest-AnalyticsBasicTestSanity-78205111:default-application-policy-set - project:default-domain:ctest-AnalyticsBasicTestSanity-78205111 , lhs: , rhs: controller/src/ifmap/ifmap_exporter.cc 574 2025-01-20 Mon 02:18:57:632.290 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754836449024, Pid 106284]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-01-20 Mon 02:18:57:632.455 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754836449024, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Deleting link controller/src/ifmap/ifmap_server_table.cc 145 2025-01-20 Mon 02:18:57:632.488 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754836449024, Pid 106284]: IFMap [SYS_DEBUG]: LinkOper: LinkRemove project:default-domain:ctest-FloatingipBasicTestSanity-53644544 - application-policy-set:default-domain:ctest-FloatingipBasicTestSanity-53644544:default-application-policy-set , lhs: , rhs: controller/src/ifmap/ifmap_exporter.cc 574 2025-01-20 Mon 02:18:57:678.356 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754819655424, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Deleting link controller/src/ifmap/ifmap_server_table.cc 145 2025-01-20 Mon 02:18:57:678.414 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754819655424, Pid 106284]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Deleting application-policy-set:default-domain:ctest-TestHeat-71270079:default-application-policy-set controller/src/ifmap/ifmap_server_table.cc 164 2025-01-20 Mon 02:18:57:678.438 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754819655424, Pid 106284]: IFMap [SYS_DEBUG]: LinkOper: LinkRemove application-policy-set:default-domain:ctest-TestHeat-71270079:default-application-policy-set - project:default-domain:ctest-TestHeat-71270079 , lhs: , rhs: controller/src/ifmap/ifmap_exporter.cc 574 2025-01-20 Mon 02:18:57:698.357 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754840647424, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Deleting link controller/src/ifmap/ifmap_server_table.cc 145 2025-01-20 Mon 02:18:57:698.409 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754840647424, Pid 106284]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Deleting project:default-domain:ctest-AnalyticsBasicTestSanity-78205111 controller/src/ifmap/ifmap_server_table.cc 164 2025-01-20 Mon 02:18:57:698.432 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754840647424, Pid 106284]: IFMap [SYS_DEBUG]: LinkOper: LinkRemove project:default-domain:ctest-AnalyticsBasicTestSanity-78205111 - domain:default-domain , lhs: , rhs: controller/src/ifmap/ifmap_exporter.cc 574 2025-01-20 Mon 02:18:57:737.987 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754819655424, Pid 106284]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-01-20 Mon 02:18:57:738.134 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754819655424, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating project-default-application-policy-set controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 02:18:57:742.017 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754861639424, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Deleting link controller/src/ifmap/ifmap_server_table.cc 145 2025-01-20 Mon 02:18:57:742.058 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754861639424, Pid 106284]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Deleting access-control-list:default-domain:ctest-TestHeat-71270079:default:egress-access-control-list controller/src/ifmap/ifmap_server_table.cc 164 2025-01-20 Mon 02:18:57:742.070 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754861639424, Pid 106284]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Deleting security-group:default-domain:ctest-TestHeat-71270079:default controller/src/ifmap/ifmap_server_table.cc 164 2025-01-20 Mon 02:18:57:742.092 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754861639424, Pid 106284]: IFMap [SYS_DEBUG]: LinkOper: LinkRemove access-control-list:default-domain:ctest-TestHeat-71270079:default:egress-access-control-list - security-group:default-domain:ctest-TestHeat-71270079:default , lhs: , rhs: controller/src/ifmap/ifmap_exporter.cc 574 2025-01-20 Mon 02:18:57:832.584 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754840647424, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Deleting link controller/src/ifmap/ifmap_server_table.cc 145 2025-01-20 Mon 02:18:57:832.622 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754840647424, Pid 106284]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Deleting application-policy-set:default-domain:ctest-FloatingipBasicTestSanity-53644544:default-application-policy-set controller/src/ifmap/ifmap_server_table.cc 164 2025-01-20 Mon 02:18:57:832.645 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754840647424, Pid 106284]: IFMap [SYS_DEBUG]: LinkOper: LinkRemove application-policy-set:default-domain:ctest-FloatingipBasicTestSanity-53644544:default-application-policy-set - project:default-domain:ctest-FloatingipBasicTestSanity-53644544 , lhs: , rhs: controller/src/ifmap/ifmap_exporter.cc 574 2025-01-20 Mon 02:18:57:860.486 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754819655424, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Deleting link controller/src/ifmap/ifmap_server_table.cc 145 2025-01-20 Mon 02:18:57:860.537 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754819655424, Pid 106284]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Deleting project:default-domain:ctest-TestHeat-71270079 controller/src/ifmap/ifmap_server_table.cc 164 2025-01-20 Mon 02:18:57:860.560 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754819655424, Pid 106284]: IFMap [SYS_DEBUG]: LinkOper: LinkRemove project:default-domain:ctest-TestHeat-71270079 - domain:default-domain , lhs: , rhs: controller/src/ifmap/ifmap_exporter.cc 574 2025-01-20 Mon 02:18:57:881.177 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754857441024, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Deleting link controller/src/ifmap/ifmap_server_table.cc 145 2025-01-20 Mon 02:18:57:881.215 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754857441024, Pid 106284]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Deleting access-control-list:default-domain:ctest-FloatingipBasicTestSanity-53644544:default:ingress-access-control-list controller/src/ifmap/ifmap_server_table.cc 164 2025-01-20 Mon 02:18:57:881.238 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754857441024, Pid 106284]: IFMap [SYS_DEBUG]: LinkOper: LinkRemove access-control-list:default-domain:ctest-FloatingipBasicTestSanity-53644544:default:ingress-access-control-list - security-group:default-domain:ctest-FloatingipBasicTestSanity-53644544:default , lhs: , rhs: controller/src/ifmap/ifmap_exporter.cc 574 2025-01-20 Mon 02:18:57:907.675 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754857441024, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Deleting link controller/src/ifmap/ifmap_server_table.cc 145 2025-01-20 Mon 02:18:57:907.718 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754857441024, Pid 106284]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Deleting project:default-domain:ctest-FloatingipBasicTestSanity-53644544 controller/src/ifmap/ifmap_server_table.cc 164 2025-01-20 Mon 02:18:57:907.740 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754857441024, Pid 106284]: IFMap [SYS_DEBUG]: LinkOper: LinkRemove project:default-domain:ctest-FloatingipBasicTestSanity-53644544 - domain:default-domain , lhs: , rhs: controller/src/ifmap/ifmap_exporter.cc 574 2025-01-20 Mon 02:18:57:950.117 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754844845824, Pid 106284]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-01-20 Mon 02:18:57:950.310 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754844845824, Pid 106284]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating security-group:default-domain:ctest-TestRoutersBasic-14108397:default controller/src/ifmap/ifmap_server_table.cc 95 2025-01-20 Mon 02:18:57:950.347 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754844845824, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating project-security-group controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 02:18:57:950.465 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754844845824, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Deleting link controller/src/ifmap/ifmap_server_table.cc 145 2025-01-20 Mon 02:18:57:950.479 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754844845824, Pid 106284]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Deleting access-control-list:default-domain:ctest-FloatingipBasicTestSanity-53644544:default:egress-access-control-list controller/src/ifmap/ifmap_server_table.cc 164 2025-01-20 Mon 02:18:57:950.489 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754844845824, Pid 106284]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Deleting security-group:default-domain:ctest-FloatingipBasicTestSanity-53644544:default controller/src/ifmap/ifmap_server_table.cc 164 2025-01-20 Mon 02:18:57:950.506 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754844845824, Pid 106284]: IFMap [SYS_DEBUG]: LinkOper: LinkRemove access-control-list:default-domain:ctest-FloatingipBasicTestSanity-53644544:default:egress-access-control-list - security-group:default-domain:ctest-FloatingipBasicTestSanity-53644544:default , lhs: , rhs: controller/src/ifmap/ifmap_exporter.cc 574 2025-01-20 Mon 02:18:58:109.152 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754836449024, Pid 106284]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-01-20 Mon 02:18:58:109.300 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754836449024, Pid 106284]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating access-control-list:default-domain:ctest-TestRoutersBasic-14108397:default:ingress-access-control-list controller/src/ifmap/ifmap_server_table.cc 95 2025-01-20 Mon 02:18:58:109.337 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754836449024, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating security-group-access-control-list controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 02:18:58:142.161 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754870036224, Pid 106284]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-01-20 Mon 02:18:58:219.340 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754832250624, Pid 106284]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-01-20 Mon 02:18:58:219.531 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754832250624, Pid 106284]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating access-control-list:default-domain:ctest-TestRoutersBasic-14108397:default:egress-access-control-list controller/src/ifmap/ifmap_server_table.cc 95 2025-01-20 Mon 02:18:58:219.571 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754832250624, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating security-group-access-control-list controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 02:19:02:269.896 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754861639424, Pid 106284]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-01-20 Mon 02:19:02:449.429 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754853242624, Pid 106284]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-01-20 Mon 02:19:02:667.497 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754861639424, Pid 106284]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-01-20 Mon 02:19:02:718.976 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754865837824, Pid 106284]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-01-20 Mon 02:19:02:754.609 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754844845824, Pid 106284]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-01-20 Mon 02:19:02:878.686 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754844845824, Pid 106284]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-01-20 Mon 02:19:03:125.211 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754853242624, Pid 106284]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-01-20 Mon 02:19:03:211.940 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754861639424, Pid 106284]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-01-20 Mon 02:19:03:212.136 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754861639424, Pid 106284]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating virtual-network:default-domain:ctest-TestRoutersBasic-14108397:ctest-vn-44164393 controller/src/ifmap/ifmap_server_table.cc 95 2025-01-20 Mon 02:19:03:212.197 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754861639424, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating project-virtual-network controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 02:19:03:273.985 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754819655424, Pid 106284]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-01-20 Mon 02:19:03:332.882 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754870036224, Pid 106284]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-01-20 Mon 02:19:03:381.074 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754840647424, Pid 106284]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-01-20 Mon 02:19:03:381.208 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754840647424, Pid 106284]: IFMap [SYS_WARN]: ConfigurationMalformedReferenceWarningLog: instance_target No entry in metadata map routing_instance e202be65-cd03-4662-ba06-5f19801a6a79 controller/src/ifmap/client/config_json_parser.cc 223 2025-01-20 Mon 02:19:03:536.206 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754823853824, Pid 106284]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-01-20 Mon 02:19:03:536.377 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754823853824, Pid 106284]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating virtual-network-network-ipam:attr(default-domain:ctest-TestRoutersBasic-14108397:ctest-vn-44164393,default-domain:default-project:default-network-ipam) controller/src/ifmap/ifmap_server_table.cc 95 2025-01-20 Mon 02:19:03:536.414 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754823853824, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating virtual-network-network-ipam controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 02:19:03:536.432 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754823853824, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating virtual-network-network-ipam controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 02:19:03:578.040 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754857441024, Pid 106284]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-01-20 Mon 02:19:03:824.291 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754836449024, Pid 106284]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-01-20 Mon 02:19:03:979.624 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754865837824, Pid 106284]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-01-20 Mon 02:19:04:058.455 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754870036224, Pid 106284]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-01-20 Mon 02:19:04:058.658 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754870036224, Pid 106284]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating virtual-network:default-domain:ctest-TestRoutersBasic-14108397:ctest-ext_vn-62634925 controller/src/ifmap/ifmap_server_table.cc 95 2025-01-20 Mon 02:19:04:058.714 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754870036224, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating project-virtual-network controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 02:19:04:139.683 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754865837824, Pid 106284]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-01-20 Mon 02:19:04:208.619 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754823853824, Pid 106284]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-01-20 Mon 02:19:04:208.804 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754865837824, Pid 106284]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-01-20 Mon 02:19:04:208.921 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754865837824, Pid 106284]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating floating-ip-pool:default-domain:ctest-TestRoutersBasic-14108397:ctest-ext_vn-62634925:floating-ip-pool controller/src/ifmap/ifmap_server_table.cc 95 2025-01-20 Mon 02:19:04:208.955 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754865837824, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating virtual-network-floating-ip-pool controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 02:19:04:275.618 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754853242624, Pid 106284]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-01-20 Mon 02:19:04:275.737 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754853242624, Pid 106284]: IFMap [SYS_WARN]: ConfigurationMalformedReferenceWarningLog: instance_target No entry in metadata map routing_instance 7543ad10-42ef-43da-8785-81162f6edc94 controller/src/ifmap/client/config_json_parser.cc 223 2025-01-20 Mon 02:19:04:403.937 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754823853824, Pid 106284]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-01-20 Mon 02:19:04:404.106 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754823853824, Pid 106284]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating virtual-network-network-ipam:attr(default-domain:ctest-TestRoutersBasic-14108397:ctest-ext_vn-62634925,default-domain:default-project:default-network-ipam) controller/src/ifmap/ifmap_server_table.cc 95 2025-01-20 Mon 02:19:04:404.130 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754823853824, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating virtual-network-network-ipam controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 02:19:04:404.154 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754823853824, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating virtual-network-network-ipam controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 02:19:04:459.585 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754823853824, Pid 106284]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-01-20 Mon 02:19:05:787.867 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754853242624, Pid 106284]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-01-20 Mon 02:19:06:731.627 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754832250624, Pid 106284]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-01-20 Mon 02:19:06:779.555 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754865837824, Pid 106284]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-01-20 Mon 02:19:06:779.998 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754865837824, Pid 106284]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating virtual-machine:b8633236-f17e-4eda-bdfc-b9362914289f controller/src/ifmap/ifmap_server_table.cc 95 2025-01-20 Mon 02:19:06:886.481 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754853242624, Pid 106284]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-01-20 Mon 02:19:06:886.688 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754853242624, Pid 106284]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating virtual-machine-interface:default-domain:ctest-TestRoutersBasic-14108397:12583078-c3b6-43ff-9176-d0db77540735 controller/src/ifmap/ifmap_server_table.cc 95 2025-01-20 Mon 02:19:06:886.731 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754853242624, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating project-virtual-machine-interface controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 02:19:06:886.756 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754853242624, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating virtual-machine-interface-security-group controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 02:19:06:886.777 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754853242624, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating virtual-machine-interface-virtual-machine controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 02:19:06:886.798 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754853242624, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating virtual-machine-interface-virtual-network controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 02:19:06:959.482 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754836449024, Pid 106284]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-01-20 Mon 02:19:07:081.727 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754857441024, Pid 106284]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-01-20 Mon 02:19:07:081.878 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754857441024, Pid 106284]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating instance-ip:728d49e5-a9e0-42bc-8f86-0ad68c3d4e6b controller/src/ifmap/ifmap_server_table.cc 95 2025-01-20 Mon 02:19:07:081.916 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754857441024, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating instance-ip-virtual-machine-interface controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 02:19:07:081.946 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754857441024, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating instance-ip-virtual-network controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 02:19:07:336.761 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754832250624, Pid 106284]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-01-20 Mon 02:19:07:404.699 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754870036224, Pid 106284]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-01-20 Mon 02:19:07:453.535 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754840647424, Pid 106284]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-01-20 Mon 02:19:07:902.226 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754828052224, Pid 106284]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-01-20 Mon 02:19:08:001.563 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754836449024, Pid 106284]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-01-20 Mon 02:19:08:001.726 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754836449024, Pid 106284]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating logical-router:default-domain:ctest-TestRoutersBasic-14108397:ctest-router1-40651829 controller/src/ifmap/ifmap_server_table.cc 95 2025-01-20 Mon 02:19:08:001.764 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754836449024, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating project-logical-router controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 02:19:08:057.271 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754857441024, Pid 106284]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-01-20 Mon 02:19:08:128.842 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754832250624, Pid 106284]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-01-20 Mon 02:19:08:129.064 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754836449024, Pid 106284]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating logical-router-virtual-network:attr(default-domain:ctest-TestRoutersBasic-14108397:ctest-ext_vn-62634925,default-domain:ctest-TestRoutersBasic-14108397:ctest-router1-40651829) controller/src/ifmap/ifmap_server_table.cc 95 2025-01-20 Mon 02:19:08:129.107 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754836449024, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating logical-router-virtual-network controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 02:19:08:129.130 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754836449024, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating logical-router-virtual-network controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 02:19:08:155.376 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754870036224, Pid 106284]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-01-20 Mon 02:19:08:313.723 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754828052224, Pid 106284]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-01-20 Mon 02:19:08:446.523 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754861639424, Pid 106284]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-01-20 Mon 02:19:08:446.706 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754861639424, Pid 106284]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating virtual-machine-interface:default-domain:ctest-TestRoutersBasic-14108397:1fa2ccca-2ed5-4735-865f-dc7cc2ecfb48 controller/src/ifmap/ifmap_server_table.cc 95 2025-01-20 Mon 02:19:08:446.747 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754861639424, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating project-virtual-machine-interface controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 02:19:08:446.786 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754861639424, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating virtual-machine-interface-virtual-network controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 02:19:08:496.326 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754836449024, Pid 106284]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-01-20 Mon 02:19:08:496.520 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754836449024, Pid 106284]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating virtual-network:default-domain:ctest-TestRoutersBasic-14108397:snat-si-left_snat_fcf145fc-7448-4efe-be87-c90391a7b7bc_eef367a7-443d-4c6a-be0a-518d5b9b1043 controller/src/ifmap/ifmap_server_table.cc 95 2025-01-20 Mon 02:19:08:496.562 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754836449024, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating project-virtual-network controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 02:19:08:496.587 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754836449024, Pid 106284]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating virtual-network-network-ipam:attr(default-domain:ctest-TestRoutersBasic-14108397:snat-si-left_snat_fcf145fc-7448-4efe-be87-c90391a7b7bc_eef367a7-443d-4c6a-be0a-518d5b9b1043,default-domain:default-project:default-network-ipam) controller/src/ifmap/ifmap_server_table.cc 95 2025-01-20 Mon 02:19:08:496.601 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754836449024, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating virtual-network-network-ipam controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 02:19:08:496.618 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754836449024, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating virtual-network-network-ipam controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 02:19:08:542.311 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754844845824, Pid 106284]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-01-20 Mon 02:19:08:620.689 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754823853824, Pid 106284]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-01-20 Mon 02:19:08:657.349 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754870036224, Pid 106284]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-01-20 Mon 02:19:08:690.880 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754836449024, Pid 106284]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-01-20 Mon 02:19:08:691.043 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754836449024, Pid 106284]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating instance-ip:84c0a579-7f44-4106-8361-6f2ded0ba367 controller/src/ifmap/ifmap_server_table.cc 95 2025-01-20 Mon 02:19:08:691.084 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754836449024, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating instance-ip-virtual-machine-interface controller/src/ifmap/ifmap_server_table.cc 130 2025-01-20 Mon 02:19:08:691.107 UTC cn-jenkins-deploy-platform-ansible-os-2650-2 [Thread 139754836449024, Pid 106284]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating instance-ip-virtual-network controller/src/ifmap/ifmap_server_table.cc 130