2025-09-11 Thu 04:17:05:184.162 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484585629824, Pid 28253]: DnsAgent [SYS_INFO]: VersionInfoLog: {"build-info":[{"build-version":"v0+nightly.r24.1.centos","build-time":"2025-09-11 00:09:15.426696","build-user":"root","build-hostname":"jenkins-build-prepare-sandbox-10925-1.novalocal","build-id":"v0+nightly.r24.1.centos-nightly_r24.1_centos.el7@contrail","build-number":"unknown"}]} src/contrail-common/base/misc_utils.cc 41 2025-09-11 Thu 04:17:05:184.299 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484585629824, Pid 28253]: SANDESH: ROLE : Generator 2025-09-11 Thu 04:17:05:184.309 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484585629824, Pid 28253]: SANDESH: MODULE : contrail-dns 2025-09-11 Thu 04:17:05:184.314 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484585629824, Pid 28253]: SANDESH: SOURCE : cn-jenkins-deploy-platform-ansible-os-4080-3 2025-09-11 Thu 04:17:05:184.319 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484585629824, Pid 28253]: SANDESH: NODE TYPE : Control 2025-09-11 Thu 04:17:05:184.323 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484585629824, Pid 28253]: SANDESH: INSTANCE ID : 0 2025-09-11 Thu 04:17:05:184.329 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484585629824, Pid 28253]: SANDESH: HTTP SERVER PORT : 8092 2025-09-11 Thu 04:17:05:184.340 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484585629824, Pid 28253]: SANDESH: System Log Send Rate Limit: 0 -> 0 2025-09-11 Thu 04:17:05:184.581 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484585629824, Pid 28253]: HTTP Introspect Init 2025-09-11 Thu 04:17:05:184.702 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484585629824, Pid 28253]: TCP [SYS_DEBUG]: TcpServerMessageLog: Server 0.0.0.0:8092 Initialization complete src/contrail-common/io/tcp_server.cc 137 2025-09-11 Thu 04:17:05:184.753 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484585629824, Pid 28253]: Sandesh Http Server Port: 8092 2025-09-11 Thu 04:17:05:184.795 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484585629824, Pid 28253]: SANDESH: NOT Writing http_port 8092TO : /tmp/contrail-dns.28133.http_port 2025-09-11 Thu 04:17:05:184.804 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484585629824, Pid 28253]: SANDESH: CONNECT TO COLLECTOR: 1 2025-09-11 Thu 04:17:05:184.890 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484585629824, Pid 28253]: Idle 2025-09-11 Thu 04:17:05:185.054 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484585629824, Pid 28253]: SANDESH: No Client: 1757564225184979 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-4080-3:Control:contrail-dns:0 client_info= [ status = Idle successful_connections = 0 pid = 28253 http_port = 8092 start_time = 1757564225184899 collector_name = collector_ip = 0.0.0.0:0 collector_list= [ [ ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 0, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 1, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 0, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-11 Thu 04:17:05:185.166 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484585629824, Pid 28253]: SANDESH: Logging: DISABLED -> ENABLED 2025-09-11 Thu 04:17:05:185.180 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484585629824, Pid 28253]: SANDESH: Logging: LEVEL: [ INVALID ] -> [ SYS_DEBUG ] log4level: [ NOTSET ] -> [ DEBUG ] 2025-09-11 Thu 04:17:05:187.716 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484425504512, Pid 28253]: Processing scm::EvStart in state Idle 2025-09-11 Thu 04:17:05:187.758 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484425504512, Pid 28253]: Disconnect 2025-09-11 Thu 04:17:05:187.827 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484425504512, Pid 28253]: Processing scm::EvCollectorUpdate in state Disconnect 2025-09-11 Thu 04:17:05:188.101 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484425504512, Pid 28253]: Connect : Start Connect timer 10.0.0.254:8086 2025-09-11 Thu 04:17:05:188.183 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484425504512, Pid 28253]: Processing scm::EvSandeshSend in state Connect 2025-09-11 Thu 04:17:05:188.264 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484425504512, Pid 28253]: SANDESH: Send FAILED: 1757564225187815 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-4080-3:Control:contrail-dns:0 client_info= [ status = Disconnect successful_connections = 0 pid = 28253 http_port = 8092 start_time = 1757564225184899 collector_name = collector_ip = 0.0.0.0:0 collector_list= [ [ ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 1, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 0, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-11 Thu 04:17:05:188.279 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484425504512, Pid 28253]: Wrong state: Connect for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-11 Thu 04:17:05:188.288 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484425504512, Pid 28253]: Processing scm::EvSandeshSend in state Connect 2025-09-11 Thu 04:17:05:188.353 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484425504512, Pid 28253]: SANDESH: Send FAILED: 1757564225187874 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-4080-3:Control:contrail-dns:0 client_info= [ status = Disconnect successful_connections = 0 pid = 28253 http_port = 8092 start_time = 1757564225184899 collector_name = collector_ip = 10.0.0.254:8086 collector_list= [ [ (*_iter40) = 10.0.0.254:8086, (*_iter40) = 10.0.0.48:8086, (*_iter40) = 10.0.0.22:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 1, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 0, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-11 Thu 04:17:05:188.362 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484425504512, Pid 28253]: Wrong state: Connect for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-11 Thu 04:17:05:188.372 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484425504512, Pid 28253]: Processing scm::EvSandeshSend in state Connect 2025-09-11 Thu 04:17:05:188.432 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484425504512, Pid 28253]: SANDESH: Send FAILED: 1757564225188174 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-4080-3:Control:contrail-dns:0 client_info= [ status = Connect successful_connections = 0 pid = 28253 http_port = 8092 start_time = 1757564225184899 collector_name = collector_ip = 10.0.0.254:8086 collector_list= [ [ (*_iter40) = 10.0.0.254:8086, (*_iter40) = 10.0.0.48:8086, (*_iter40) = 10.0.0.22:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 1, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 0, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-11 Thu 04:17:05:188.440 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484425504512, Pid 28253]: Wrong state: Connect for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-11 Thu 04:17:05:378.615 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484585629824, Pid 28253]: TCP [SYS_DEBUG]: TcpServerMessageLog: Server 10.20.0.14:8093 Initialization complete src/contrail-common/io/tcp_server.cc 137 2025-09-11 Thu 04:17:05:378.811 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484585629824, Pid 28253]: DnsAgent [SYS_NOTICE]: DnsOperational: Bulk sync has been started controller/src/dns/main.cc 380 2025-09-11 Thu 04:17:05:378.880 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484328527616, Pid 28253]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Config Client Mgr SM: Start RabbitMqReader and init Database src/contrail-common/config-client-mgr/config_client_manager.cc 295 2025-09-11 Thu 04:17:05:380.844 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484328527616, Pid 28253]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: RabbitMQ SM: RabbitMQ connection down src/contrail-common/config-client-mgr/config_amqp_client.cc 159 2025-09-11 Thu 04:17:05:380.863 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484349519616, Pid 28253]: Processing scm::EvSandeshSend in state Connect 2025-09-11 Thu 04:17:05:380.915 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484349519616, Pid 28253]: SANDESH: Send FAILED: 1757564225380817 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-4080-3 process_status= [ [ [ module_id = contrail-dns instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.254:8086, ] ] status = Initializing description = Disconnect : EvCollectorUpdate -> Connect ], [ type = Database name = RabbitMQ server_addrs= [ [ (*_iter6) = 10.0.0.254:5673, (*_iter6) = 10.0.0.22:5673, (*_iter6) = 10.0.0.48:5673, ] ] status = Down description = RabbitMQ connection down ], ] ] flag_infos= [ [ ] ] description = Number of connections:2, Expected:3 Missing: Database:Cassandra ], ] ] ] 2025-09-11 Thu 04:17:05:380.926 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484349519616, Pid 28253]: Wrong state: Connect for event: EvSandeshSend message: NodeStatusUVE 2025-09-11 Thu 04:17:05:380.870 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484328527616, Pid 28253]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: RabbitMQ SM: Connect to Rabbit MQ with queue_delete TRUE src/contrail-common/config-client-mgr/config_amqp_client.cc 168 2025-09-11 Thu 04:17:05:474.806 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484328527616, Pid 28253]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: RabbitMQ SM: RabbitMQ connection established src/contrail-common/config-client-mgr/config_amqp_client.cc 152 2025-09-11 Thu 04:17:05:474.842 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484341122816, Pid 28253]: Processing scm::EvSandeshSend in state Connect 2025-09-11 Thu 04:17:05:474.896 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484341122816, Pid 28253]: SANDESH: Send FAILED: 1757564225474763 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-4080-3 process_status= [ [ [ module_id = contrail-dns instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.254:8086, ] ] status = Initializing description = Disconnect : EvCollectorUpdate -> Connect ], [ type = Database name = RabbitMQ server_addrs= [ [ (*_iter6) = 10.0.0.254:5673, (*_iter6) = 10.0.0.22:5673, (*_iter6) = 10.0.0.48:5673, ] ] status = Up description = RabbitMQ connection established ], ] ] flag_infos= [ [ ] ] description = Number of connections:2, Expected:3 Missing: Database:Cassandra ], ] ] ] 2025-09-11 Thu 04:17:05:474.907 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484341122816, Pid 28253]: Wrong state: Connect for event: EvSandeshSend message: NodeStatusUVE 2025-09-11 Thu 04:17:05:474.846 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484328527616, Pid 28253]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Config Client Mgr SM: Init Database src/contrail-common/config-client-mgr/config_client_manager.cc 300 2025-09-11 Thu 04:17:05:475.372 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484328527616, Pid 28253]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Lost Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 578 2025-09-11 Thu 04:17:05:475.383 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484328527616, Pid 28253]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Db Init src/contrail-common/config-client-mgr/config_cassandra_client.cc 85 2025-09-11 Thu 04:17:05:475.388 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484349519616, Pid 28253]: Processing scm::EvSandeshSend in state Connect 2025-09-11 Thu 04:17:05:476.801 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484349519616, Pid 28253]: SANDESH: Send FAILED: 1757564225475341 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-4080-3 process_status= [ [ [ module_id = contrail-dns instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.254:8086, ] ] status = Initializing description = Disconnect : EvCollectorUpdate -> Connect ], [ type = Database name = Cassandra server_addrs= [ [ (*_iter6) = 10.0.0.254:9041, (*_iter6) = 10.0.0.22:9041, (*_iter6) = 10.0.0.48:9041, ] ] status = Down description = Lost Cassandra connection ], [ type = Database name = RabbitMQ server_addrs= [ [ (*_iter6) = 10.0.0.254:5673, (*_iter6) = 10.0.0.22:5673, (*_iter6) = 10.0.0.48:5673, ] ] status = Up description = RabbitMQ connection established ], ] ] flag_infos= [ [ ] ] description = Collector, Database:Cassandra connection down ], ] ] ] 2025-09-11 Thu 04:17:05:476.812 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484349519616, Pid 28253]: Wrong state: Connect for event: EvSandeshSend message: NodeStatusUVE 2025-09-11 Thu 04:17:05:503.188 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484328527616, Pid 28253]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Db SetTableSpace src/contrail-common/config-client-mgr/config_cassandra_client.cc 93 2025-09-11 Thu 04:17:05:504.235 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484328527616, Pid 28253]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Db UseColumnFamily uuidTableName src/contrail-common/config-client-mgr/config_cassandra_client.cc 102 2025-09-11 Thu 04:17:05:504.280 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484328527616, Pid 28253]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Db UseColumnFamily fqnTableName src/contrail-common/config-client-mgr/config_cassandra_client.cc 108 2025-09-11 Thu 04:17:05:504.348 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484328527616, Pid 28253]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 571 2025-09-11 Thu 04:17:05:504.358 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484328527616, Pid 28253]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: BulkDataSync Started src/contrail-common/config-client-mgr/config_cassandra_client.cc 391 2025-09-11 Thu 04:17:05:504.375 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484328527616, Pid 28253]: Processing scm::EvSandeshSend in state Connect 2025-09-11 Thu 04:17:05:504.404 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484328527616, Pid 28253]: SANDESH: Send FAILED: 1757564225504315 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-4080-3 process_status= [ [ [ module_id = contrail-dns instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.254:8086, ] ] status = Initializing description = Disconnect : EvCollectorUpdate -> Connect ], [ type = Database name = Cassandra server_addrs= [ [ (*_iter6) = 10.0.0.254:9041, (*_iter6) = 10.0.0.22:9041, (*_iter6) = 10.0.0.48:9041, ] ] status = Up description = Established Cassandra connection ], [ type = Database name = RabbitMQ server_addrs= [ [ (*_iter6) = 10.0.0.254:5673, (*_iter6) = 10.0.0.22:5673, (*_iter6) = 10.0.0.48:5673, ] ] status = Up description = RabbitMQ connection established ], ] ] flag_infos= [ [ ] ] description = Collector connection down ], ] ] ] 2025-09-11 Thu 04:17:05:504.419 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484328527616, Pid 28253]: Wrong state: Connect for event: EvSandeshSend message: NodeStatusUVE 2025-09-11 Thu 04:17:05:507.852 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484341122816, Pid 28253]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 571 2025-09-11 Thu 04:17:05:512.577 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484341122816, Pid 28253]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 571 2025-09-11 Thu 04:17:05:514.872 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484341122816, Pid 28253]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 571 2025-09-11 Thu 04:17:05:522.973 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484341122816, Pid 28253]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 571 2025-09-11 Thu 04:17:05:524.927 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484341122816, Pid 28253]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 571 2025-09-11 Thu 04:17:05:526.039 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484341122816, Pid 28253]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 571 2025-09-11 Thu 04:17:05:527.747 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484341122816, Pid 28253]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 571 2025-09-11 Thu 04:17:05:529.724 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484341122816, Pid 28253]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 571 2025-09-11 Thu 04:17:05:532.186 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484341122816, Pid 28253]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 571 2025-09-11 Thu 04:17:05:542.690 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484341122816, Pid 28253]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 571 2025-09-11 Thu 04:17:05:548.249 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484341122816, Pid 28253]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 571 2025-09-11 Thu 04:17:05:559.038 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484341122816, Pid 28253]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 571 2025-09-11 Thu 04:17:05:567.498 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484341122816, Pid 28253]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 571 2025-09-11 Thu 04:17:05:569.656 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484341122816, Pid 28253]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 571 2025-09-11 Thu 04:17:05:574.244 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484341122816, Pid 28253]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 571 2025-09-11 Thu 04:17:05:577.258 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484341122816, Pid 28253]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 571 2025-09-11 Thu 04:17:05:583.710 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484341122816, Pid 28253]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 571 2025-09-11 Thu 04:17:05:587.881 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484341122816, Pid 28253]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 571 2025-09-11 Thu 04:17:05:592.419 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484341122816, Pid 28253]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 571 2025-09-11 Thu 04:17:05:598.166 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484341122816, Pid 28253]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 571 2025-09-11 Thu 04:17:05:602.843 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484341122816, Pid 28253]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 571 2025-09-11 Thu 04:17:05:607.667 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484341122816, Pid 28253]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 571 2025-09-11 Thu 04:17:05:613.149 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484341122816, Pid 28253]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 571 2025-09-11 Thu 04:17:05:617.693 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484341122816, Pid 28253]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 571 2025-09-11 Thu 04:17:05:624.127 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484341122816, Pid 28253]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 571 2025-09-11 Thu 04:17:05:626.356 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484341122816, Pid 28253]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 571 2025-09-11 Thu 04:17:05:627.279 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484341122816, Pid 28253]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 571 2025-09-11 Thu 04:17:05:635.776 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484341122816, Pid 28253]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 571 2025-09-11 Thu 04:17:05:637.144 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484341122816, Pid 28253]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 571 2025-09-11 Thu 04:17:05:638.591 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484341122816, Pid 28253]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 571 2025-09-11 Thu 04:17:05:653.091 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484341122816, Pid 28253]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 571 2025-09-11 Thu 04:17:05:655.676 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484341122816, Pid 28253]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 571 2025-09-11 Thu 04:17:05:665.418 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484341122816, Pid 28253]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 571 2025-09-11 Thu 04:17:05:670.885 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484341122816, Pid 28253]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 571 2025-09-11 Thu 04:17:05:682.277 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484341122816, Pid 28253]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 571 2025-09-11 Thu 04:17:05:687.436 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484341122816, Pid 28253]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 571 2025-09-11 Thu 04:17:05:696.620 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484341122816, Pid 28253]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 571 2025-09-11 Thu 04:17:05:713.859 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484341122816, Pid 28253]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 571 2025-09-11 Thu 04:17:05:715.146 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484341122816, Pid 28253]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 571 2025-09-11 Thu 04:17:05:716.520 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484341122816, Pid 28253]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 571 2025-09-11 Thu 04:17:05:721.951 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484341122816, Pid 28253]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 571 2025-09-11 Thu 04:17:05:723.437 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484341122816, Pid 28253]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 571 2025-09-11 Thu 04:17:05:731.095 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484341122816, Pid 28253]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 571 2025-09-11 Thu 04:17:05:733.840 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484341122816, Pid 28253]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 571 2025-09-11 Thu 04:17:05:738.257 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484341122816, Pid 28253]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 571 2025-09-11 Thu 04:17:05:751.294 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484341122816, Pid 28253]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 571 2025-09-11 Thu 04:17:05:753.473 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484341122816, Pid 28253]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 571 2025-09-11 Thu 04:17:05:754.425 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484341122816, Pid 28253]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 571 2025-09-11 Thu 04:17:05:765.460 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484341122816, Pid 28253]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 571 2025-09-11 Thu 04:17:05:766.828 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484341122816, Pid 28253]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 571 2025-09-11 Thu 04:17:05:773.329 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484341122816, Pid 28253]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 571 2025-09-11 Thu 04:17:05:776.642 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484341122816, Pid 28253]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 571 2025-09-11 Thu 04:17:05:777.843 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484341122816, Pid 28253]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 571 2025-09-11 Thu 04:17:05:786.972 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484341122816, Pid 28253]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 571 2025-09-11 Thu 04:17:05:792.447 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484341122816, Pid 28253]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 571 2025-09-11 Thu 04:17:05:793.549 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484341122816, Pid 28253]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 571 2025-09-11 Thu 04:17:05:794.351 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484341122816, Pid 28253]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 571 2025-09-11 Thu 04:17:05:798.760 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484341122816, Pid 28253]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 571 2025-09-11 Thu 04:17:05:803.743 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484341122816, Pid 28253]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 571 2025-09-11 Thu 04:17:05:804.946 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484341122816, Pid 28253]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 571 2025-09-11 Thu 04:17:05:817.568 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484341122816, Pid 28253]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 571 2025-09-11 Thu 04:17:05:820.206 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484341122816, Pid 28253]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 571 2025-09-11 Thu 04:17:05:831.888 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484341122816, Pid 28253]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 571 2025-09-11 Thu 04:17:05:841.903 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484341122816, Pid 28253]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 571 2025-09-11 Thu 04:17:05:844.180 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484341122816, Pid 28253]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 571 2025-09-11 Thu 04:17:05:851.363 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484341122816, Pid 28253]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 571 2025-09-11 Thu 04:17:05:876.757 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484341122816, Pid 28253]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 571 2025-09-11 Thu 04:17:05:884.347 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484341122816, Pid 28253]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 571 2025-09-11 Thu 04:17:05:886.408 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484341122816, Pid 28253]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 571 2025-09-11 Thu 04:17:05:899.632 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484341122816, Pid 28253]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 571 2025-09-11 Thu 04:17:05:901.904 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484341122816, Pid 28253]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 571 2025-09-11 Thu 04:17:05:904.746 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484341122816, Pid 28253]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 571 2025-09-11 Thu 04:17:05:921.953 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484341122816, Pid 28253]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 571 2025-09-11 Thu 04:17:05:926.239 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484341122816, Pid 28253]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 571 2025-09-11 Thu 04:17:05:927.496 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484341122816, Pid 28253]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 571 2025-09-11 Thu 04:17:05:935.482 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484341122816, Pid 28253]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 571 2025-09-11 Thu 04:17:05:938.608 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484341122816, Pid 28253]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 571 2025-09-11 Thu 04:17:05:939.623 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484341122816, Pid 28253]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 571 2025-09-11 Thu 04:17:05:943.228 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484341122816, Pid 28253]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 571 2025-09-11 Thu 04:17:05:945.801 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484341122816, Pid 28253]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 571 2025-09-11 Thu 04:17:05:950.747 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484341122816, Pid 28253]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 571 2025-09-11 Thu 04:17:05:958.355 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484341122816, Pid 28253]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 571 2025-09-11 Thu 04:17:05:960.247 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484341122816, Pid 28253]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 571 2025-09-11 Thu 04:17:05:967.143 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484341122816, Pid 28253]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 571 2025-09-11 Thu 04:17:06:002.524 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484341122816, Pid 28253]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 571 2025-09-11 Thu 04:17:06:004.510 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484341122816, Pid 28253]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 571 2025-09-11 Thu 04:17:06:006.751 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484341122816, Pid 28253]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 571 2025-09-11 Thu 04:17:06:010.841 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484341122816, Pid 28253]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 571 2025-09-11 Thu 04:17:06:013.733 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484341122816, Pid 28253]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 571 2025-09-11 Thu 04:17:06:015.492 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484341122816, Pid 28253]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 571 2025-09-11 Thu 04:17:06:045.121 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484341122816, Pid 28253]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 571 2025-09-11 Thu 04:17:06:047.201 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484341122816, Pid 28253]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 571 2025-09-11 Thu 04:17:06:048.168 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484341122816, Pid 28253]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 571 2025-09-11 Thu 04:17:06:066.893 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484341122816, Pid 28253]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 571 2025-09-11 Thu 04:17:06:072.551 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484341122816, Pid 28253]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 571 2025-09-11 Thu 04:17:06:073.375 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484341122816, Pid 28253]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 571 2025-09-11 Thu 04:17:06:084.862 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484341122816, Pid 28253]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 571 2025-09-11 Thu 04:17:06:091.127 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484341122816, Pid 28253]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 571 2025-09-11 Thu 04:17:06:093.532 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484341122816, Pid 28253]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 571 2025-09-11 Thu 04:17:06:104.764 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484341122816, Pid 28253]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 571 2025-09-11 Thu 04:17:06:111.224 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484341122816, Pid 28253]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 571 2025-09-11 Thu 04:17:06:122.569 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484341122816, Pid 28253]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 571 2025-09-11 Thu 04:17:06:270.880 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484341122816, Pid 28253]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 571 2025-09-11 Thu 04:17:06:273.031 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484341122816, Pid 28253]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 571 2025-09-11 Thu 04:17:06:276.772 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484341122816, Pid 28253]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 571 2025-09-11 Thu 04:17:06:280.880 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484341122816, Pid 28253]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 571 2025-09-11 Thu 04:17:06:285.268 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484341122816, Pid 28253]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 571 2025-09-11 Thu 04:17:06:286.758 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484341122816, Pid 28253]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 571 2025-09-11 Thu 04:17:06:304.011 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484341122816, Pid 28253]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 571 2025-09-11 Thu 04:17:06:342.691 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484341122816, Pid 28253]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 571 2025-09-11 Thu 04:17:06:350.959 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484341122816, Pid 28253]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 571 2025-09-11 Thu 04:17:06:363.410 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484341122816, Pid 28253]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 571 2025-09-11 Thu 04:17:06:374.084 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484341122816, Pid 28253]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 571 2025-09-11 Thu 04:17:06:379.696 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484341122816, Pid 28253]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 571 2025-09-11 Thu 04:17:06:403.586 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484341122816, Pid 28253]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 571 2025-09-11 Thu 04:17:06:418.795 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484341122816, Pid 28253]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 571 2025-09-11 Thu 04:17:06:420.154 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484341122816, Pid 28253]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 571 2025-09-11 Thu 04:17:06:441.752 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484341122816, Pid 28253]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 571 2025-09-11 Thu 04:17:06:442.853 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484332726016, Pid 28253]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: One reader finished BulkSync src/contrail-common/config-client-mgr/config_cassandra_client.cc 556 2025-09-11 Thu 04:17:06:442.868 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484425504512, Pid 28253]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: One reader finished BulkSync src/contrail-common/config-client-mgr/config_cassandra_client.cc 556 2025-09-11 Thu 04:17:06:442.875 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484349519616, Pid 28253]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: One reader finished BulkSync src/contrail-common/config-client-mgr/config_cassandra_client.cc 556 2025-09-11 Thu 04:17:06:442.876 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484341122816, Pid 28253]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: One reader finished BulkSync src/contrail-common/config-client-mgr/config_cassandra_client.cc 556 2025-09-11 Thu 04:17:06:442.891 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484324329216, Pid 28253]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: One reader finished BulkSync src/contrail-common/config-client-mgr/config_cassandra_client.cc 556 2025-09-11 Thu 04:17:06:442.910 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484320130816, Pid 28253]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: One reader finished BulkSync src/contrail-common/config-client-mgr/config_cassandra_client.cc 556 2025-09-11 Thu 04:17:06:442.925 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484336924416, Pid 28253]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: One reader finished BulkSync src/contrail-common/config-client-mgr/config_cassandra_client.cc 556 2025-09-11 Thu 04:17:06:442.938 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484353718016, Pid 28253]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: BulkSyncDone by all readers src/contrail-common/config-client-mgr/config_cassandra_client.cc 552 2025-09-11 Thu 04:17:06:443.314 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484353718016, Pid 28253]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Config Client Mgr SM: End of RIB computed and notification sent src/contrail-common/config-client-mgr/config_client_manager.cc 191 2025-09-11 Thu 04:17:06:443.418 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484311734016, Pid 28253]: Processing scm::EvSandeshSend in state Connect 2025-09-11 Thu 04:17:06:443.448 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484311734016, Pid 28253]: SANDESH: Send FAILED: 1757564226443343 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-4080-3 process_status= [ [ [ module_id = contrail-dns instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.254:8086, ] ] status = Initializing description = Disconnect : EvCollectorUpdate -> Connect ], [ type = Database name = Cassandra server_addrs= [ [ (*_iter6) = 10.0.0.254:9041, (*_iter6) = 10.0.0.22:9041, (*_iter6) = 10.0.0.48:9041, ] ] status = Up description = Established Cassandra connection ], [ type = Database name = RabbitMQ server_addrs= [ [ (*_iter6) = 10.0.0.254:5673, (*_iter6) = 10.0.0.22:5673, (*_iter6) = 10.0.0.48:5673, ] ] status = Up description = RabbitMQ connection established ], ] ] flag_infos= [ [ ] ] description = Collector connection down ], ] ] ] 2025-09-11 Thu 04:17:06:443.457 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484311734016, Pid 28253]: Wrong state: Connect for event: EvSandeshSend message: NodeStatusUVE 2025-09-11 Thu 04:17:06:443.539 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484328527616, Pid 28253]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: TRUE src/contrail-common/config-client-mgr/config_client_manager.cc 219 2025-09-11 Thu 04:17:06:448.550 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484341122816, Pid 28253]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 571 2025-09-11 Thu 04:17:06:448.728 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484311734016, Pid 28253]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating tag-type:application controller/src/ifmap/ifmap_server_table.cc 95 2025-09-11 Thu 04:17:06:448.805 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484311734016, Pid 28253]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating config-database-node:default-global-system-config:cn-jenkins-deploy-platform-ansible-os-4080-1. controller/src/ifmap/ifmap_server_table.cc 95 2025-09-11 Thu 04:17:06:448.839 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484311734016, Pid 28253]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating global-system-config:default-global-system-config controller/src/ifmap/ifmap_server_table.cc 95 2025-09-11 Thu 04:17:06:448.856 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484311734016, Pid 28253]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-config-database-node controller/src/ifmap/ifmap_server_table.cc 130 2025-09-11 Thu 04:17:06:448.885 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484311734016, Pid 28253]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating virtual-network:default-domain:default-project:dci-network controller/src/ifmap/ifmap_server_table.cc 95 2025-09-11 Thu 04:17:06:448.922 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484311734016, Pid 28253]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating project:default-domain:default-project controller/src/ifmap/ifmap_server_table.cc 95 2025-09-11 Thu 04:17:06:448.934 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484311734016, Pid 28253]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating project-virtual-network controller/src/ifmap/ifmap_server_table.cc 130 2025-09-11 Thu 04:17:06:448.969 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484311734016, Pid 28253]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating api-access-list:default-global-system-config:default-api-access-list controller/src/ifmap/ifmap_server_table.cc 95 2025-09-11 Thu 04:17:06:448.996 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484311734016, Pid 28253]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-api-access-list controller/src/ifmap/ifmap_server_table.cc 130 2025-09-11 Thu 04:17:06:449.009 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484311734016, Pid 28253]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating alarm:default-global-system-config:system-defined-conf-incorrect controller/src/ifmap/ifmap_server_table.cc 95 2025-09-11 Thu 04:17:06:449.028 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484311734016, Pid 28253]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-alarm controller/src/ifmap/ifmap_server_table.cc 130 2025-09-11 Thu 04:17:06:449.043 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484311734016, Pid 28253]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating security-group:default-domain:default-project:__no_rule__ controller/src/ifmap/ifmap_server_table.cc 95 2025-09-11 Thu 04:17:06:449.061 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484311734016, Pid 28253]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating project-security-group controller/src/ifmap/ifmap_server_table.cc 130 2025-09-11 Thu 04:17:06:449.432 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484425504512, Pid 28253]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 571 2025-09-11 Thu 04:17:06:449.770 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484425504512, Pid 28253]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating alarm:default-global-system-config:system-defined-core-files controller/src/ifmap/ifmap_server_table.cc 95 2025-09-11 Thu 04:17:06:449.803 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484425504512, Pid 28253]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-alarm controller/src/ifmap/ifmap_server_table.cc 130 2025-09-11 Thu 04:17:06:449.818 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484425504512, Pid 28253]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating network-ipam:default-domain:default-project:default-network-ipam controller/src/ifmap/ifmap_server_table.cc 95 2025-09-11 Thu 04:17:06:449.838 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484425504512, Pid 28253]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating project-network-ipam controller/src/ifmap/ifmap_server_table.cc 130 2025-09-11 Thu 04:17:06:449.851 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484425504512, Pid 28253]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating policy-management:default-policy-management controller/src/ifmap/ifmap_server_table.cc 95 2025-09-11 Thu 04:17:06:449.865 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484425504512, Pid 28253]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating alarm:default-global-system-config:system-defined-bottle-request-size-limit controller/src/ifmap/ifmap_server_table.cc 95 2025-09-11 Thu 04:17:06:449.879 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484425504512, Pid 28253]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-alarm controller/src/ifmap/ifmap_server_table.cc 130 2025-09-11 Thu 04:17:06:449.903 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484425504512, Pid 28253]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating domain:default-domain controller/src/ifmap/ifmap_server_table.cc 95 2025-09-11 Thu 04:17:06:449.913 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484425504512, Pid 28253]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating domain-project controller/src/ifmap/ifmap_server_table.cc 130 2025-09-11 Thu 04:17:06:449.929 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484425504512, Pid 28253]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating alarm:default-global-system-config:system-defined-address-mismatch-compute controller/src/ifmap/ifmap_server_table.cc 95 2025-09-11 Thu 04:17:06:449.954 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484425504512, Pid 28253]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-alarm controller/src/ifmap/ifmap_server_table.cc 130 2025-09-11 Thu 04:17:06:450.767 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484349519616, Pid 28253]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 571 2025-09-11 Thu 04:17:06:451.292 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484349519616, Pid 28253]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating network-ipam:default-domain:default-project:service-chain-flat-ipam controller/src/ifmap/ifmap_server_table.cc 95 2025-09-11 Thu 04:17:06:451.318 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484349519616, Pid 28253]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating project-network-ipam controller/src/ifmap/ifmap_server_table.cc 130 2025-09-11 Thu 04:17:06:451.332 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484349519616, Pid 28253]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating virtual-network:default-domain:default-project:__link_local__ controller/src/ifmap/ifmap_server_table.cc 95 2025-09-11 Thu 04:17:06:451.360 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484349519616, Pid 28253]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating project-virtual-network controller/src/ifmap/ifmap_server_table.cc 130 2025-09-11 Thu 04:17:06:451.373 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484349519616, Pid 28253]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating alarm:default-global-system-config:system-defined-prouter-tsn-connectivity controller/src/ifmap/ifmap_server_table.cc 95 2025-09-11 Thu 04:17:06:451.390 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484349519616, Pid 28253]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-alarm controller/src/ifmap/ifmap_server_table.cc 130 2025-09-11 Thu 04:17:06:451.404 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484349519616, Pid 28253]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating tag-type:tier controller/src/ifmap/ifmap_server_table.cc 95 2025-09-11 Thu 04:17:06:451.417 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484349519616, Pid 28253]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating job-template:default-global-system-config:maintenance_mode_activate_template controller/src/ifmap/ifmap_server_table.cc 95 2025-09-11 Thu 04:17:06:451.451 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484349519616, Pid 28253]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-job-template controller/src/ifmap/ifmap_server_table.cc 130 2025-09-11 Thu 04:17:06:451.463 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484349519616, Pid 28253]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating namespace:default-domain:default-namespace controller/src/ifmap/ifmap_server_table.cc 95 2025-09-11 Thu 04:17:06:451.477 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484349519616, Pid 28253]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating domain-namespace controller/src/ifmap/ifmap_server_table.cc 130 2025-09-11 Thu 04:17:06:451.488 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484349519616, Pid 28253]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating job-template:default-global-system-config:device_import_template controller/src/ifmap/ifmap_server_table.cc 95 2025-09-11 Thu 04:17:06:451.505 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484349519616, Pid 28253]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-job-template controller/src/ifmap/ifmap_server_table.cc 130 2025-09-11 Thu 04:17:06:451.981 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484336924416, Pid 28253]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 571 2025-09-11 Thu 04:17:06:452.684 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484336924416, Pid 28253]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating alarm:default-global-system-config:system-defined-storage-cluster-state controller/src/ifmap/ifmap_server_table.cc 95 2025-09-11 Thu 04:17:06:452.710 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484336924416, Pid 28253]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-alarm controller/src/ifmap/ifmap_server_table.cc 130 2025-09-11 Thu 04:17:06:452.725 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484336924416, Pid 28253]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating job-template:default-global-system-config:discover_role_template controller/src/ifmap/ifmap_server_table.cc 95 2025-09-11 Thu 04:17:06:452.746 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484336924416, Pid 28253]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-job-template controller/src/ifmap/ifmap_server_table.cc 130 2025-09-11 Thu 04:17:06:452.759 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484336924416, Pid 28253]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating alarm:default-global-system-config:system-defined-vrouter-limit-exceeded controller/src/ifmap/ifmap_server_table.cc 95 2025-09-11 Thu 04:17:06:452.788 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484336924416, Pid 28253]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-alarm controller/src/ifmap/ifmap_server_table.cc 130 2025-09-11 Thu 04:17:06:452.800 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484336924416, Pid 28253]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating job-template:default-global-system-config:ztp_template controller/src/ifmap/ifmap_server_table.cc 95 2025-09-11 Thu 04:17:06:452.818 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484336924416, Pid 28253]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-job-template controller/src/ifmap/ifmap_server_table.cc 130 2025-09-11 Thu 04:17:06:452.831 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484336924416, Pid 28253]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating alarm:default-global-system-config:system-defined-node-status controller/src/ifmap/ifmap_server_table.cc 95 2025-09-11 Thu 04:17:06:452.853 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484336924416, Pid 28253]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-alarm controller/src/ifmap/ifmap_server_table.cc 130 2025-09-11 Thu 04:17:06:452.865 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484336924416, Pid 28253]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating virtual-network:default-domain:default-project:ip-fabric controller/src/ifmap/ifmap_server_table.cc 95 2025-09-11 Thu 04:17:06:452.892 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484336924416, Pid 28253]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating project-virtual-network controller/src/ifmap/ifmap_server_table.cc 130 2025-09-11 Thu 04:17:06:452.905 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484336924416, Pid 28253]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating tag-type:deployment controller/src/ifmap/ifmap_server_table.cc 95 2025-09-11 Thu 04:17:06:452.915 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484336924416, Pid 28253]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating service-template:default-domain:analyzer-template controller/src/ifmap/ifmap_server_table.cc 95 2025-09-11 Thu 04:17:06:452.931 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484336924416, Pid 28253]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating domain-service-template controller/src/ifmap/ifmap_server_table.cc 130 2025-09-11 Thu 04:17:06:452.943 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484336924416, Pid 28253]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating alarm:default-global-system-config:system-defined-vrouter-interface controller/src/ifmap/ifmap_server_table.cc 95 2025-09-11 Thu 04:17:06:452.964 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484336924416, Pid 28253]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-alarm controller/src/ifmap/ifmap_server_table.cc 130 2025-09-11 Thu 04:17:06:452.976 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484336924416, Pid 28253]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating alarm:default-global-system-config:system-defined-disk-usage-high controller/src/ifmap/ifmap_server_table.cc 95 2025-09-11 Thu 04:17:06:452.990 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484336924416, Pid 28253]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-alarm controller/src/ifmap/ifmap_server_table.cc 130 2025-09-11 Thu 04:17:06:453.001 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484336924416, Pid 28253]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating job-template:default-global-system-config:discover_device_template controller/src/ifmap/ifmap_server_table.cc 95 2025-09-11 Thu 04:17:06:453.018 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484336924416, Pid 28253]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-job-template controller/src/ifmap/ifmap_server_table.cc 130 2025-09-11 Thu 04:17:06:453.030 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484336924416, Pid 28253]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating global-qos-config:default-global-system-config:default-global-qos-config controller/src/ifmap/ifmap_server_table.cc 95 2025-09-11 Thu 04:17:06:453.042 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484336924416, Pid 28253]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-global-qos-config controller/src/ifmap/ifmap_server_table.cc 130 2025-09-11 Thu 04:17:06:453.053 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484336924416, Pid 28253]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating alarm:default-global-system-config:system-defined-process-status controller/src/ifmap/ifmap_server_table.cc 95 2025-09-11 Thu 04:17:06:453.067 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484336924416, Pid 28253]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-alarm controller/src/ifmap/ifmap_server_table.cc 130 2025-09-11 Thu 04:17:06:453.480 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484353718016, Pid 28253]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 571 2025-09-11 Thu 04:17:06:453.827 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484353718016, Pid 28253]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating virtual-network:default-domain:default-project:default-virtual-network controller/src/ifmap/ifmap_server_table.cc 95 2025-09-11 Thu 04:17:06:453.851 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484353718016, Pid 28253]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating project-virtual-network controller/src/ifmap/ifmap_server_table.cc 130 2025-09-11 Thu 04:17:06:453.866 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484353718016, Pid 28253]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating tag-type:neutron_tag controller/src/ifmap/ifmap_server_table.cc 95 2025-09-11 Thu 04:17:06:453.877 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484353718016, Pid 28253]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating config-node:default-global-system-config:cn-jenkins-deploy-platform-ansible-os-4080-1. controller/src/ifmap/ifmap_server_table.cc 95 2025-09-11 Thu 04:17:06:453.893 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484353718016, Pid 28253]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-config-node controller/src/ifmap/ifmap_server_table.cc 130 2025-09-11 Thu 04:17:06:453.910 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484353718016, Pid 28253]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating application-policy-set:default-policy-management:default-application-policy-set controller/src/ifmap/ifmap_server_table.cc 95 2025-09-11 Thu 04:17:06:453.926 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484353718016, Pid 28253]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating policy-management-application-policy-set controller/src/ifmap/ifmap_server_table.cc 130 2025-09-11 Thu 04:17:06:453.940 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484353718016, Pid 28253]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating global-vrouter-config:default-global-system-config:default-global-vrouter-config controller/src/ifmap/ifmap_server_table.cc 95 2025-09-11 Thu 04:17:06:453.952 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484353718016, Pid 28253]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating application-policy-set-global-vrouter-config controller/src/ifmap/ifmap_server_table.cc 130 2025-09-11 Thu 04:17:06:453.964 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484353718016, Pid 28253]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating alarm:default-global-system-config:system-defined-pending-cassandra-compaction-tasks controller/src/ifmap/ifmap_server_table.cc 95 2025-09-11 Thu 04:17:06:453.980 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484353718016, Pid 28253]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-alarm controller/src/ifmap/ifmap_server_table.cc 130 2025-09-11 Thu 04:17:06:454.001 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484353718016, Pid 28253]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-global-vrouter-config controller/src/ifmap/ifmap_server_table.cc 130 2025-09-11 Thu 04:17:06:454.013 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484353718016, Pid 28253]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating discovery-service-assignment:default-discovery-service-assignment controller/src/ifmap/ifmap_server_table.cc 95 2025-09-11 Thu 04:17:06:454.024 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484353718016, Pid 28253]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating alarm:default-global-system-config:system-defined-xmpp-connectivity controller/src/ifmap/ifmap_server_table.cc 95 2025-09-11 Thu 04:17:06:454.041 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484353718016, Pid 28253]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-alarm controller/src/ifmap/ifmap_server_table.cc 130 2025-09-11 Thu 04:17:06:456.348 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484332726016, Pid 28253]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 571 2025-09-11 Thu 04:17:06:456.851 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484332726016, Pid 28253]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating alarm:default-global-system-config:system-defined-partial-sysinfo controller/src/ifmap/ifmap_server_table.cc 95 2025-09-11 Thu 04:17:06:456.877 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484320130816, Pid 28253]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 571 2025-09-11 Thu 04:17:06:456.879 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484332726016, Pid 28253]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-alarm controller/src/ifmap/ifmap_server_table.cc 130 2025-09-11 Thu 04:17:06:456.931 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484332726016, Pid 28253]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating service-template:default-domain:haproxy-loadbalancer-template controller/src/ifmap/ifmap_server_table.cc 95 2025-09-11 Thu 04:17:06:456.966 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484332726016, Pid 28253]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating domain-service-template controller/src/ifmap/ifmap_server_table.cc 130 2025-09-11 Thu 04:17:06:457.021 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484332726016, Pid 28253]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating alarm:default-global-system-config:system-defined-package-version-mismatch controller/src/ifmap/ifmap_server_table.cc 95 2025-09-11 Thu 04:17:06:457.041 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484332726016, Pid 28253]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-alarm controller/src/ifmap/ifmap_server_table.cc 130 2025-09-11 Thu 04:17:06:457.054 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484332726016, Pid 28253]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating alarm:default-global-system-config:system-defined-disk-usage-critical controller/src/ifmap/ifmap_server_table.cc 95 2025-09-11 Thu 04:17:06:457.070 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484332726016, Pid 28253]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-alarm controller/src/ifmap/ifmap_server_table.cc 130 2025-09-11 Thu 04:17:06:457.082 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484332726016, Pid 28253]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating alarm:default-global-system-config:system-defined-prouter-connectivity controller/src/ifmap/ifmap_server_table.cc 95 2025-09-11 Thu 04:17:06:457.138 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484341122816, Pid 28253]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-alarm controller/src/ifmap/ifmap_server_table.cc 130 2025-09-11 Thu 04:17:06:457.169 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484341122816, Pid 28253]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating alarm:default-global-system-config:system-defined-conf-elements-inconsistency controller/src/ifmap/ifmap_server_table.cc 95 2025-09-11 Thu 04:17:06:457.189 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484341122816, Pid 28253]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-alarm controller/src/ifmap/ifmap_server_table.cc 130 2025-09-11 Thu 04:17:06:457.202 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484341122816, Pid 28253]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating tag-type:site controller/src/ifmap/ifmap_server_table.cc 95 2025-09-11 Thu 04:17:06:457.215 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484341122816, Pid 28253]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating alarm:default-global-system-config:system-defined-vrouter-l3mh-physical-interface-link-down controller/src/ifmap/ifmap_server_table.cc 95 2025-09-11 Thu 04:17:06:457.231 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484341122816, Pid 28253]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-alarm controller/src/ifmap/ifmap_server_table.cc 130 2025-09-11 Thu 04:17:06:457.244 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484341122816, Pid 28253]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating config-node:default-global-system-config:cn-jenkins-deploy-platform-ansible-os-4080-3. controller/src/ifmap/ifmap_server_table.cc 95 2025-09-11 Thu 04:17:06:457.259 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484341122816, Pid 28253]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-config-node controller/src/ifmap/ifmap_server_table.cc 130 2025-09-11 Thu 04:17:06:457.272 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484341122816, Pid 28253]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating config-database-node:default-global-system-config:cn-jenkins-deploy-platform-ansible-os-4080-2. controller/src/ifmap/ifmap_server_table.cc 95 2025-09-11 Thu 04:17:06:457.286 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484341122816, Pid 28253]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-config-database-node controller/src/ifmap/ifmap_server_table.cc 130 2025-09-11 Thu 04:17:06:457.308 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484341122816, Pid 28253]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating job-template:default-global-system-config:cli_sync_template controller/src/ifmap/ifmap_server_table.cc 95 2025-09-11 Thu 04:17:06:457.352 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484341122816, Pid 28253]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-job-template controller/src/ifmap/ifmap_server_table.cc 130 2025-09-11 Thu 04:17:06:457.404 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484341122816, Pid 28253]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating job-template:default-global-system-config:fabric_config_template controller/src/ifmap/ifmap_server_table.cc 95 2025-09-11 Thu 04:17:06:457.427 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484341122816, Pid 28253]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-job-template controller/src/ifmap/ifmap_server_table.cc 130 2025-09-11 Thu 04:17:06:457.441 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484341122816, Pid 28253]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating service-template:default-domain:netns-snat-template controller/src/ifmap/ifmap_server_table.cc 95 2025-09-11 Thu 04:17:06:457.460 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484341122816, Pid 28253]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating domain-service-template controller/src/ifmap/ifmap_server_table.cc 130 2025-09-11 Thu 04:17:06:457.489 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484341122816, Pid 28253]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating job-template:default-global-system-config:container_cleanup_template controller/src/ifmap/ifmap_server_table.cc 95 2025-09-11 Thu 04:17:06:457.512 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484341122816, Pid 28253]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-job-template controller/src/ifmap/ifmap_server_table.cc 130 2025-09-11 Thu 04:17:06:457.556 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484341122816, Pid 28253]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating tag-type:label controller/src/ifmap/ifmap_server_table.cc 95 2025-09-11 Thu 04:17:06:457.614 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484429702912, Pid 28253]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating alarm:default-global-system-config:system-defined-address-mismatch-control controller/src/ifmap/ifmap_server_table.cc 95 2025-09-11 Thu 04:17:06:457.660 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484429702912, Pid 28253]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-alarm controller/src/ifmap/ifmap_server_table.cc 130 2025-09-11 Thu 04:17:06:457.692 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484341122816, Pid 28253]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating job-template:default-global-system-config:maintenance_mode_deactivate_template controller/src/ifmap/ifmap_server_table.cc 95 2025-09-11 Thu 04:17:06:457.717 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484341122816, Pid 28253]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-job-template controller/src/ifmap/ifmap_server_table.cc 130 2025-09-11 Thu 04:17:06:457.731 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484341122816, Pid 28253]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating config-database-node:default-global-system-config:cn-jenkins-deploy-platform-ansible-os-4080-3. controller/src/ifmap/ifmap_server_table.cc 95 2025-09-11 Thu 04:17:06:457.748 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484341122816, Pid 28253]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-config-database-node controller/src/ifmap/ifmap_server_table.cc 130 2025-09-11 Thu 04:17:06:457.766 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484341122816, Pid 28253]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating alarm:default-global-system-config:system-defined-xmpp-close-reason controller/src/ifmap/ifmap_server_table.cc 95 2025-09-11 Thu 04:17:06:457.787 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484341122816, Pid 28253]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-alarm controller/src/ifmap/ifmap_server_table.cc 130 2025-09-11 Thu 04:17:06:457.801 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484341122816, Pid 28253]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating config-node:default-global-system-config:cn-jenkins-deploy-platform-ansible-os-4080-2. controller/src/ifmap/ifmap_server_table.cc 95 2025-09-11 Thu 04:17:06:457.816 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484341122816, Pid 28253]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-config-node controller/src/ifmap/ifmap_server_table.cc 130 2025-09-11 Thu 04:17:06:457.829 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484341122816, Pid 28253]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating tag-type:neutron_fwaas controller/src/ifmap/ifmap_server_table.cc 95 2025-09-11 Thu 04:17:06:457.964 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484429702912, Pid 28253]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating service-template:default-domain:docker-template controller/src/ifmap/ifmap_server_table.cc 95 2025-09-11 Thu 04:17:06:457.988 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484429702912, Pid 28253]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating domain-service-template controller/src/ifmap/ifmap_server_table.cc 130 2025-09-11 Thu 04:17:06:458.005 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484429702912, Pid 28253]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating alarm:default-global-system-config:system-defined-bgp-connectivity controller/src/ifmap/ifmap_server_table.cc 95 2025-09-11 Thu 04:17:06:458.026 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484429702912, Pid 28253]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-alarm controller/src/ifmap/ifmap_server_table.cc 130 2025-09-11 Thu 04:17:06:458.044 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484429702912, Pid 28253]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating service-appliance-set:default-global-system-config:native controller/src/ifmap/ifmap_server_table.cc 95 2025-09-11 Thu 04:17:06:458.064 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484429702912, Pid 28253]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-service-appliance-set controller/src/ifmap/ifmap_server_table.cc 130 2025-09-11 Thu 04:17:06:458.078 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484429702912, Pid 28253]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating job-template:default-global-system-config:discover_os_computes_template controller/src/ifmap/ifmap_server_table.cc 95 2025-09-11 Thu 04:17:06:458.107 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484429702912, Pid 28253]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-job-template controller/src/ifmap/ifmap_server_table.cc 130 2025-09-11 Thu 04:17:06:458.121 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484429702912, Pid 28253]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating alarm:default-global-system-config:system-defined-process-connectivity controller/src/ifmap/ifmap_server_table.cc 95 2025-09-11 Thu 04:17:06:458.141 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484429702912, Pid 28253]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-alarm controller/src/ifmap/ifmap_server_table.cc 130 2025-09-11 Thu 04:17:06:458.155 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484429702912, Pid 28253]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating service-template:default-domain:nat-template controller/src/ifmap/ifmap_server_table.cc 95 2025-09-11 Thu 04:17:06:458.188 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484429702912, Pid 28253]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating domain-service-template controller/src/ifmap/ifmap_server_table.cc 130 2025-09-11 Thu 04:17:06:458.553 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484324329216, Pid 28253]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 571 2025-09-11 Thu 04:17:06:458.654 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484429702912, Pid 28253]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating application-policy-set:default-domain:default-project:default-application-policy-set controller/src/ifmap/ifmap_server_table.cc 95 2025-09-11 Thu 04:17:06:458.683 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484429702912, Pid 28253]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating project-application-policy-set controller/src/ifmap/ifmap_server_table.cc 130 2025-09-11 Thu 04:17:06:458.700 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484429702912, Pid 28253]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating alarm:default-global-system-config:system-defined-vrouter-table-limit-exceeded controller/src/ifmap/ifmap_server_table.cc 95 2025-09-11 Thu 04:17:06:458.723 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484429702912, Pid 28253]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-alarm controller/src/ifmap/ifmap_server_table.cc 130 2025-09-11 Thu 04:17:06:458.782 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484349519616, Pid 28253]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating job-template:default-global-system-config:hitless_upgrade_strategy_template controller/src/ifmap/ifmap_server_table.cc 95 2025-09-11 Thu 04:17:06:458.809 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484349519616, Pid 28253]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-job-template controller/src/ifmap/ifmap_server_table.cc 130 2025-09-11 Thu 04:17:06:458.842 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484349519616, Pid 28253]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating job-template:default-global-system-config:image_upgrade_template controller/src/ifmap/ifmap_server_table.cc 95 2025-09-11 Thu 04:17:06:458.866 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484349519616, Pid 28253]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-job-template controller/src/ifmap/ifmap_server_table.cc 130 2025-09-11 Thu 04:17:06:458.908 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484328527616, Pid 28253]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating fabric:default-global-system-config:default-fabric controller/src/ifmap/ifmap_server_table.cc 95 2025-09-11 Thu 04:17:06:458.927 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484328527616, Pid 28253]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-fabric controller/src/ifmap/ifmap_server_table.cc 130 2025-09-11 Thu 04:17:06:458.954 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484328527616, Pid 28253]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating service-appliance-set:default-global-system-config:opencontrail controller/src/ifmap/ifmap_server_table.cc 95 2025-09-11 Thu 04:17:06:458.972 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484328527616, Pid 28253]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-service-appliance-set controller/src/ifmap/ifmap_server_table.cc 130 2025-09-11 Thu 04:17:06:459.737 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484585629824, Pid 28253]: DnsAgent [SYS_NOTICE]: DnsOperational: EndOfRib point was reached, IF-MAP objects have been created controller/src/dns/main.cc 86 2025-09-11 Thu 04:17:06:459.761 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484585629824, Pid 28253]: DnsAgent [SYS_NOTICE]: DnsOperational: The IF-MAP database table has been updated: __ifmap__.virtual_DNS.0, counting 0 elements controller/src/dns/main.cc 98 2025-09-11 Thu 04:17:06:459.777 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484585629824, Pid 28253]: DnsAgent [SYS_NOTICE]: DnsOperational: The IF-MAP database table has been updated: __ifmap__.virtual_DNS_record.0, counting 0 elements controller/src/dns/main.cc 98 2025-09-11 Thu 04:17:06:459.786 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484585629824, Pid 28253]: DnsAgent [SYS_NOTICE]: DnsOperational: The IF-MAP database table has been updated: __ifmap__.network_ipam.0, counting 2 elements controller/src/dns/main.cc 98 2025-09-11 Thu 04:17:06:459.794 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484585629824, Pid 28253]: DnsAgent [SYS_NOTICE]: DnsOperational: The IF-MAP database table has been updated: __ifmap__.virtual_network_network_ipam.0, counting 0 elements controller/src/dns/main.cc 98 2025-09-11 Thu 04:17:06:459.802 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484585629824, Pid 28253]: DnsAgent [SYS_NOTICE]: DnsOperational: The IF-MAP database table has been updated: __ifmap__.global_qos_config.0, counting 1 elements controller/src/dns/main.cc 98 2025-09-11 Thu 04:17:06:459.810 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484585629824, Pid 28253]: DnsAgent [SYS_NOTICE]: DnsOperational: VnniConfig has been updated after the bulk sync, the number of elements: 0 controller/src/dns/main.cc 109 2025-09-11 Thu 04:17:06:459.817 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484585629824, Pid 28253]: DnsAgent [SYS_NOTICE]: DnsOperational: IpamConfig has been updated after the bulk sync, the number of elements: 2 controller/src/dns/main.cc 120 2025-09-11 Thu 04:17:06:459.825 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484585629824, Pid 28253]: DnsAgent [SYS_NOTICE]: DnsOperational: VirtualDnsConfig has been updated after the bulk sync, the number of elements: 0 controller/src/dns/main.cc 131 2025-09-11 Thu 04:17:06:459.832 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484585629824, Pid 28253]: DnsAgent [SYS_NOTICE]: DnsOperational: VirtualDnsRecordConfig has been updated after the bulk sync, the number of elements: 0 controller/src/dns/main.cc 142 2025-09-11 Thu 04:17:06:459.839 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484585629824, Pid 28253]: DnsAgent [SYS_NOTICE]: DnsOperational: GlobalQosConfig has been updated after the bulk sync controller/src/dns/main.cc 152 2025-09-11 Thu 04:17:06:459.901 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484585629824, Pid 28253]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: RabbitMQ SM: RabbitMQ connection down src/contrail-common/config-client-mgr/config_amqp_client.cc 159 2025-09-11 Thu 04:17:06:459.911 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484585629824, Pid 28253]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: RabbitMQ SM: Connect to Rabbit MQ with queue_delete TRUE src/contrail-common/config-client-mgr/config_amqp_client.cc 168 2025-09-11 Thu 04:17:06:459.918 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484336924416, Pid 28253]: Processing scm::EvSandeshSend in state Connect 2025-09-11 Thu 04:17:06:459.970 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484336924416, Pid 28253]: SANDESH: Send FAILED: 1757564226459869 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-4080-3 process_status= [ [ [ module_id = contrail-dns instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.254:8086, ] ] status = Initializing description = Disconnect : EvCollectorUpdate -> Connect ], [ type = Database name = Cassandra server_addrs= [ [ (*_iter6) = 10.0.0.254:9041, (*_iter6) = 10.0.0.22:9041, (*_iter6) = 10.0.0.48:9041, ] ] status = Up description = Established Cassandra connection ], [ type = Database name = RabbitMQ server_addrs= [ [ (*_iter6) = 10.0.0.254:5673, (*_iter6) = 10.0.0.22:5673, (*_iter6) = 10.0.0.48:5673, ] ] status = Down description = RabbitMQ connection down ], ] ] flag_infos= [ [ ] ] description = Collector, Database:RabbitMQ connection down ], ] ] ] 2025-09-11 Thu 04:17:06:459.985 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484336924416, Pid 28253]: Wrong state: Connect for event: EvSandeshSend message: NodeStatusUVE 2025-09-11 Thu 04:17:06:720.209 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484336924416, Pid 28253]: Processing scm::EvSandeshSend in state Connect 2025-09-11 Thu 04:17:06:720.265 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484336924416, Pid 28253]: SANDESH: Send FAILED: 1757564226719941 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-4080-3 process_status= [ [ [ module_id = contrail-dns instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.254:8086, ] ] status = Initializing description = Disconnect : EvCollectorUpdate -> Connect ], [ type = Database name = Cassandra server_addrs= [ [ (*_iter6) = 10.0.0.254:9041, (*_iter6) = 10.0.0.22:9041, (*_iter6) = 10.0.0.48:9041, ] ] status = Up description = Established Cassandra connection ], [ type = Database name = RabbitMQ server_addrs= [ [ (*_iter6) = 10.0.0.254:5673, (*_iter6) = 10.0.0.22:5673, (*_iter6) = 10.0.0.48:5673, ] ] status = Up description = RabbitMQ connection established ], ] ] flag_infos= [ [ ] ] description = Collector connection down ], ] ] ] 2025-09-11 Thu 04:17:06:720.275 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484336924416, Pid 28253]: Wrong state: Connect for event: EvSandeshSend message: NodeStatusUVE 2025-09-11 Thu 04:17:06:720.298 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484585629824, Pid 28253]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: RabbitMQ SM: RabbitMQ connection established src/contrail-common/config-client-mgr/config_amqp_client.cc 152 2025-09-11 Thu 04:17:06:720.317 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484336924416, Pid 28253]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: TRUE src/contrail-common/config-client-mgr/config_client_manager.cc 219 2025-09-11 Thu 04:17:06:720.785 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484585629824, Pid 28253]: -(-1) OnSessionEvent TCP Connect Failed 2025-09-11 Thu 04:17:06:720.794 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484585629824, Pid 28253]: -(-1):EvTcpConnectFail 2025-09-11 Thu 04:17:06:721.198 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484349519616, Pid 28253]: Processing scm::EvTcpConnectFail in state Connect 2025-09-11 Thu 04:17:06:721.208 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484349519616, Pid 28253]: Connect : EvTcpConnectFail 2025-09-11 Thu 04:17:06:721.355 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484349519616, Pid 28253]: TCP [SYS_ERR]: TcpSessionMessageLog: Session -(-1) > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 312 2025-09-11 Thu 04:17:06:721.390 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484349519616, Pid 28253]: Idle 2025-09-11 Thu 04:17:06:721.448 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484349519616, Pid 28253]: Processing scm::EvSandeshSend in state Idle 2025-09-11 Thu 04:17:06:721.594 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484349519616, Pid 28253]: SANDESH: Send FAILED: 1757564226721312 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-4080-3:Control:contrail-dns:0 client_info= [ status = Connect successful_connections = 0 pid = 28253 http_port = 8092 start_time = 1757564225184899 collector_name = collector_ip = 10.0.0.48:8086 collector_list= [ [ (*_iter40) = 10.0.0.254:8086, (*_iter40) = 10.0.0.48:8086, (*_iter40) = 10.0.0.22:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ConfigClientMgrDebug [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 152 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = DnsOperational [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 12 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapLinkOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 65 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapNodeOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 75 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 7 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 3 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 306, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 10, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-11 Thu 04:17:06:721.616 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484349519616, Pid 28253]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-11 Thu 04:17:06:721.625 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484349519616, Pid 28253]: Processing scm::EvSandeshSend in state Idle 2025-09-11 Thu 04:17:06:721.655 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484349519616, Pid 28253]: SANDESH: Send FAILED: 1757564226721333 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-4080-3 process_status= [ [ [ module_id = contrail-dns instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], [ type = Database name = Cassandra server_addrs= [ [ (*_iter6) = 10.0.0.254:9041, (*_iter6) = 10.0.0.22:9041, (*_iter6) = 10.0.0.48:9041, ] ] status = Up description = Established Cassandra connection ], [ type = Database name = RabbitMQ server_addrs= [ [ (*_iter6) = 10.0.0.254:5673, (*_iter6) = 10.0.0.22:5673, (*_iter6) = 10.0.0.48:5673, ] ] status = Up description = RabbitMQ connection established ], ] ] flag_infos= [ [ ] ] description = Collector connection down ], ] ] ] 2025-09-11 Thu 04:17:06:721.662 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484349519616, Pid 28253]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-11 Thu 04:17:06:721.670 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484349519616, Pid 28253]: Processing scm::EvTcpDeleteSession in state Idle 2025-09-11 Thu 04:17:06:721.678 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484349519616, Pid 28253]: Processing scm::EvSandeshSend in state Idle 2025-09-11 Thu 04:17:06:721.806 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484349519616, Pid 28253]: SANDESH: Send FAILED: 1757564226721439 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-4080-3:Control:contrail-dns:0 client_info= [ status = Idle successful_connections = 0 pid = 28253 http_port = 8092 start_time = 1757564225184899 collector_name = collector_ip = 10.0.0.48:8086 collector_list= [ [ (*_iter40) = 10.0.0.254:8086, (*_iter40) = 10.0.0.48:8086, (*_iter40) = 10.0.0.22:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ConfigClientMgrDebug [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 152 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = DnsOperational [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 12 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapLinkOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 65 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapNodeOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 75 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 7 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 3 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 307, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 10, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-11 Thu 04:17:06:721.819 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484349519616, Pid 28253]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-11 Thu 04:17:06:722.186 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484429702912, Pid 28253]: DnsAgent [SYS_NOTICE]: DnsOperational: BIND named up; DNS is operational controller/src/dns/mgr/dns_mgr.cc 737 2025-09-11 Thu 04:17:07:721.945 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484425504512, Pid 28253]: Processing scm::EvIdleHoldTimerExpired in state Idle 2025-09-11 Thu 04:17:07:722.134 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484425504512, Pid 28253]: Connect : Start Connect timer 10.0.0.48:8086 2025-09-11 Thu 04:17:07:722.155 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484585629824, Pid 28253]: -(-1) OnSessionEvent TCP Connect Failed 2025-09-11 Thu 04:17:07:722.179 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484585629824, Pid 28253]: -(-1):EvTcpConnectFail 2025-09-11 Thu 04:17:07:722.233 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484425504512, Pid 28253]: Processing scm::EvSandeshSend in state Connect 2025-09-11 Thu 04:17:07:722.261 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484425504512, Pid 28253]: SANDESH: Send FAILED: 1757564227722000 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-4080-3 process_status= [ [ [ module_id = contrail-dns instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.48:8086, ] ] status = Initializing description = Idle : EvIdleHoldTimerExpired -> Connect ], [ type = Database name = Cassandra server_addrs= [ [ (*_iter6) = 10.0.0.254:9041, (*_iter6) = 10.0.0.22:9041, (*_iter6) = 10.0.0.48:9041, ] ] status = Up description = Established Cassandra connection ], [ type = Database name = RabbitMQ server_addrs= [ [ (*_iter6) = 10.0.0.254:5673, (*_iter6) = 10.0.0.22:5673, (*_iter6) = 10.0.0.48:5673, ] ] status = Up description = RabbitMQ connection established ], ] ] flag_infos= [ [ ] ] description = Collector connection down ], ] ] ] 2025-09-11 Thu 04:17:07:722.270 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484425504512, Pid 28253]: Wrong state: Connect for event: EvSandeshSend message: NodeStatusUVE 2025-09-11 Thu 04:17:07:722.283 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484425504512, Pid 28253]: Processing scm::EvTcpConnectFail in state Connect 2025-09-11 Thu 04:17:07:722.290 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484425504512, Pid 28253]: Connect : EvTcpConnectFail 2025-09-11 Thu 04:17:07:722.378 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484425504512, Pid 28253]: TCP [SYS_ERR]: TcpSessionMessageLog: Session -(-1) > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 312 2025-09-11 Thu 04:17:07:722.402 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484425504512, Pid 28253]: Idle 2025-09-11 Thu 04:17:07:722.457 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484425504512, Pid 28253]: Processing scm::EvSandeshSend in state Idle 2025-09-11 Thu 04:17:07:722.611 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484425504512, Pid 28253]: SANDESH: Send FAILED: 1757564227722220 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-4080-3:Control:contrail-dns:0 client_info= [ status = Connect successful_connections = 0 pid = 28253 http_port = 8092 start_time = 1757564225184899 collector_name = collector_ip = 10.0.0.48:8086 collector_list= [ [ (*_iter40) = 10.0.0.254:8086, (*_iter40) = 10.0.0.48:8086, (*_iter40) = 10.0.0.22:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ConfigClientMgrDebug [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 152 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = DnsOperational [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 13 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapLinkOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 65 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapNodeOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 75 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 8 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 5 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 308, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 13, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-11 Thu 04:17:07:722.629 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484425504512, Pid 28253]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-11 Thu 04:17:07:722.652 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484425504512, Pid 28253]: Processing scm::EvSandeshSend in state Idle 2025-09-11 Thu 04:17:07:722.773 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484425504512, Pid 28253]: SANDESH: Send FAILED: 1757564227722336 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-4080-3:Control:contrail-dns:0 client_info= [ status = Connect successful_connections = 0 pid = 28253 http_port = 8092 start_time = 1757564225184899 collector_name = collector_ip = 10.0.0.22:8086 collector_list= [ [ (*_iter40) = 10.0.0.254:8086, (*_iter40) = 10.0.0.48:8086, (*_iter40) = 10.0.0.22:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ConfigClientMgrDebug [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 152 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = DnsOperational [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 13 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapLinkOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 65 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapNodeOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 75 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 9 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 5 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 308, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 14, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-11 Thu 04:17:07:722.786 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484425504512, Pid 28253]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-11 Thu 04:17:07:722.794 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484425504512, Pid 28253]: Processing scm::EvSandeshSend in state Idle 2025-09-11 Thu 04:17:07:722.814 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484425504512, Pid 28253]: SANDESH: Send FAILED: 1757564227722356 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-4080-3 process_status= [ [ [ module_id = contrail-dns instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], [ type = Database name = Cassandra server_addrs= [ [ (*_iter6) = 10.0.0.254:9041, (*_iter6) = 10.0.0.22:9041, (*_iter6) = 10.0.0.48:9041, ] ] status = Up description = Established Cassandra connection ], [ type = Database name = RabbitMQ server_addrs= [ [ (*_iter6) = 10.0.0.254:5673, (*_iter6) = 10.0.0.22:5673, (*_iter6) = 10.0.0.48:5673, ] ] status = Up description = RabbitMQ connection established ], ] ] flag_infos= [ [ ] ] description = Collector connection down ], ] ] ] 2025-09-11 Thu 04:17:07:722.820 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484425504512, Pid 28253]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-11 Thu 04:17:07:722.827 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484425504512, Pid 28253]: Processing scm::EvTcpDeleteSession in state Idle 2025-09-11 Thu 04:17:07:722.844 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484425504512, Pid 28253]: Processing scm::EvSandeshSend in state Idle 2025-09-11 Thu 04:17:07:722.964 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484425504512, Pid 28253]: SANDESH: Send FAILED: 1757564227722448 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-4080-3:Control:contrail-dns:0 client_info= [ status = Idle successful_connections = 0 pid = 28253 http_port = 8092 start_time = 1757564225184899 collector_name = collector_ip = 10.0.0.22:8086 collector_list= [ [ (*_iter40) = 10.0.0.254:8086, (*_iter40) = 10.0.0.48:8086, (*_iter40) = 10.0.0.22:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ConfigClientMgrDebug [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 152 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = DnsOperational [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 13 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapLinkOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 65 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapNodeOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 75 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 9 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 5 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 309, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 14, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-11 Thu 04:17:07:722.978 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484425504512, Pid 28253]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-11 Thu 04:17:07:829.147 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484315932416, Pid 28253]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 571 2025-09-11 Thu 04:17:07:915.978 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484429702912, Pid 28253]: DnsAgent [SYS_NOTICE]: DnsOperational: Bulk sync has been done (the DB has been synced with named) controller/src/dns/mgr/dns_mgr.cc 455 2025-09-11 Thu 04:17:09:722.512 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484341122816, Pid 28253]: Processing scm::EvIdleHoldTimerExpired in state Idle 2025-09-11 Thu 04:17:09:722.799 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484341122816, Pid 28253]: Connect : Start Connect timer 10.0.0.22:8086 2025-09-11 Thu 04:17:09:722.960 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484341122816, Pid 28253]: Processing scm::EvSandeshSend in state Connect 2025-09-11 Thu 04:17:09:722.993 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484341122816, Pid 28253]: SANDESH: Send FAILED: 1757564229722613 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-4080-3 process_status= [ [ [ module_id = contrail-dns instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.22:8086, ] ] status = Initializing description = Idle : EvIdleHoldTimerExpired -> Connect ], [ type = Database name = Cassandra server_addrs= [ [ (*_iter6) = 10.0.0.254:9041, (*_iter6) = 10.0.0.22:9041, (*_iter6) = 10.0.0.48:9041, ] ] status = Up description = Established Cassandra connection ], [ type = Database name = RabbitMQ server_addrs= [ [ (*_iter6) = 10.0.0.254:5673, (*_iter6) = 10.0.0.22:5673, (*_iter6) = 10.0.0.48:5673, ] ] status = Up description = RabbitMQ connection established ], ] ] flag_infos= [ [ ] ] description = Collector connection down ], ] ] ] 2025-09-11 Thu 04:17:09:723.007 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484341122816, Pid 28253]: Wrong state: Connect for event: EvSandeshSend message: NodeStatusUVE 2025-09-11 Thu 04:17:09:723.016 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484341122816, Pid 28253]: Processing scm::EvSandeshSend in state Connect 2025-09-11 Thu 04:17:09:723.096 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484585629824, Pid 28253]: -(-1) OnSessionEvent TCP Connect Failed 2025-09-11 Thu 04:17:09:723.129 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484585629824, Pid 28253]: -(-1):EvTcpConnectFail 2025-09-11 Thu 04:17:09:723.158 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484341122816, Pid 28253]: SANDESH: Send FAILED: 1757564229722940 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-4080-3:Control:contrail-dns:0 client_info= [ status = Connect successful_connections = 0 pid = 28253 http_port = 8092 start_time = 1757564225184899 collector_name = collector_ip = 10.0.0.22:8086 collector_list= [ [ (*_iter40) = 10.0.0.254:8086, (*_iter40) = 10.0.0.48:8086, (*_iter40) = 10.0.0.22:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ConfigClientMgrDebug [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 153 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = DnsOperational [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 14 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapLinkOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 65 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapNodeOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 75 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 10 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 8 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 311, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 18, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-11 Thu 04:17:09:723.174 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484341122816, Pid 28253]: Wrong state: Connect for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-11 Thu 04:17:09:723.185 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484341122816, Pid 28253]: Processing scm::EvTcpConnectFail in state Connect 2025-09-11 Thu 04:17:09:723.194 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484341122816, Pid 28253]: Connect : EvTcpConnectFail 2025-09-11 Thu 04:17:09:723.288 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484341122816, Pid 28253]: TCP [SYS_ERR]: TcpSessionMessageLog: Session -(-1) > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 312 2025-09-11 Thu 04:17:09:723.319 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484341122816, Pid 28253]: Idle 2025-09-11 Thu 04:17:09:723.380 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484341122816, Pid 28253]: Processing scm::EvSandeshSend in state Idle 2025-09-11 Thu 04:17:09:723.518 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484341122816, Pid 28253]: SANDESH: Send FAILED: 1757564229723244 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-4080-3:Control:contrail-dns:0 client_info= [ status = Connect successful_connections = 0 pid = 28253 http_port = 8092 start_time = 1757564225184899 collector_name = collector_ip = 10.0.0.254:8086 collector_list= [ [ (*_iter40) = 10.0.0.254:8086, (*_iter40) = 10.0.0.48:8086, (*_iter40) = 10.0.0.22:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ConfigClientMgrDebug [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 153 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = DnsOperational [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 14 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapLinkOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 65 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapNodeOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 75 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 11 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 9 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 311, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 20, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-11 Thu 04:17:09:723.532 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484341122816, Pid 28253]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-11 Thu 04:17:09:723.542 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484341122816, Pid 28253]: Processing scm::EvSandeshSend in state Idle 2025-09-11 Thu 04:17:09:723.563 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484341122816, Pid 28253]: SANDESH: Send FAILED: 1757564229723265 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-4080-3 process_status= [ [ [ module_id = contrail-dns instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], [ type = Database name = Cassandra server_addrs= [ [ (*_iter6) = 10.0.0.254:9041, (*_iter6) = 10.0.0.22:9041, (*_iter6) = 10.0.0.48:9041, ] ] status = Up description = Established Cassandra connection ], [ type = Database name = RabbitMQ server_addrs= [ [ (*_iter6) = 10.0.0.254:5673, (*_iter6) = 10.0.0.22:5673, (*_iter6) = 10.0.0.48:5673, ] ] status = Up description = RabbitMQ connection established ], ] ] flag_infos= [ [ ] ] description = Collector connection down ], ] ] ] 2025-09-11 Thu 04:17:09:723.572 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484341122816, Pid 28253]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-11 Thu 04:17:09:723.580 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484341122816, Pid 28253]: Processing scm::EvTcpDeleteSession in state Idle 2025-09-11 Thu 04:17:09:723.598 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484341122816, Pid 28253]: Processing scm::EvSandeshSend in state Idle 2025-09-11 Thu 04:17:09:723.736 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484341122816, Pid 28253]: SANDESH: Send FAILED: 1757564229723366 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-4080-3:Control:contrail-dns:0 client_info= [ status = Idle successful_connections = 0 pid = 28253 http_port = 8092 start_time = 1757564225184899 collector_name = collector_ip = 10.0.0.254:8086 collector_list= [ [ (*_iter40) = 10.0.0.254:8086, (*_iter40) = 10.0.0.48:8086, (*_iter40) = 10.0.0.22:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ConfigClientMgrDebug [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 153 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = DnsOperational [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 14 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapLinkOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 65 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapNodeOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 75 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 11 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 9 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 312, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 20, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-11 Thu 04:17:09:723.749 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484341122816, Pid 28253]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-11 Thu 04:17:10:870.922 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484311734016, Pid 28253]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 571 2025-09-11 Thu 04:17:10:871.057 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484311734016, Pid 28253]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating access-control-list:default-domain:default-project:__no_rule__:ingress-access-control-list controller/src/ifmap/ifmap_server_table.cc 95 2025-09-11 Thu 04:17:10:871.100 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484311734016, Pid 28253]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating security-group-access-control-list controller/src/ifmap/ifmap_server_table.cc 130 2025-09-11 Thu 04:17:10:924.125 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484353718016, Pid 28253]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 571 2025-09-11 Thu 04:17:10:924.266 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484353718016, Pid 28253]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating access-control-list:default-domain:default-project:__no_rule__:egress-access-control-list controller/src/ifmap/ifmap_server_table.cc 95 2025-09-11 Thu 04:17:10:924.298 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484353718016, Pid 28253]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating security-group-access-control-list controller/src/ifmap/ifmap_server_table.cc 130 2025-09-11 Thu 04:17:10:996.236 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484320130816, Pid 28253]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 571 2025-09-11 Thu 04:17:11:019.188 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484311734016, Pid 28253]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 571 2025-09-11 Thu 04:17:11:019.292 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484311734016, Pid 28253]: IFMap [SYS_WARN]: ConfigurationMalformedReferenceWarningLog: instance_target No entry in metadata map routing_instance 2e1fc63d-6d97-43e5-861a-2c3093d27091 controller/src/ifmap/client/config_json_parser.cc 223 2025-09-11 Thu 04:17:11:061.802 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484429702912, Pid 28253]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 571 2025-09-11 Thu 04:17:11:088.022 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484311734016, Pid 28253]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 571 2025-09-11 Thu 04:17:11:088.126 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484311734016, Pid 28253]: IFMap [SYS_WARN]: ConfigurationMalformedReferenceWarningLog: instance_target No entry in metadata map routing_instance a79654b7-d507-44f7-bbb7-2b6bf4c79ce3 controller/src/ifmap/client/config_json_parser.cc 223 2025-09-11 Thu 04:17:11:123.060 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484353718016, Pid 28253]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 571 2025-09-11 Thu 04:17:11:169.597 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484324329216, Pid 28253]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 571 2025-09-11 Thu 04:17:11:169.719 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484324329216, Pid 28253]: IFMap [SYS_WARN]: ConfigurationMalformedReferenceWarningLog: instance_target No entry in metadata map routing_instance 85522317-b3f9-447d-bea5-72a8afc7db3b controller/src/ifmap/client/config_json_parser.cc 223 2025-09-11 Thu 04:17:11:227.149 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484425504512, Pid 28253]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 571 2025-09-11 Thu 04:17:11:252.669 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484320130816, Pid 28253]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 571 2025-09-11 Thu 04:17:11:252.762 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484320130816, Pid 28253]: IFMap [SYS_WARN]: ConfigurationMalformedReferenceWarningLog: instance_target No entry in metadata map routing_instance 010c7f62-733c-4f41-aaaa-9712d470b6e4 controller/src/ifmap/client/config_json_parser.cc 223 2025-09-11 Thu 04:17:11:383.801 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484332726016, Pid 28253]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 571 2025-09-11 Thu 04:17:11:383.962 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484332726016, Pid 28253]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating access-control-list:default-domain:default-project:ip-fabric:ip-fabric controller/src/ifmap/ifmap_server_table.cc 95 2025-09-11 Thu 04:17:11:384.002 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484332726016, Pid 28253]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating virtual-network-access-control-list controller/src/ifmap/ifmap_server_table.cc 130 2025-09-11 Thu 04:17:11:416.925 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484345321216, Pid 28253]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 571 2025-09-11 Thu 04:17:11:442.568 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484353718016, Pid 28253]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 571 2025-09-11 Thu 04:17:13:025.375 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484341122816, Pid 28253]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 571 2025-09-11 Thu 04:17:13:084.599 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484429702912, Pid 28253]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 571 2025-09-11 Thu 04:17:13:113.844 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484429702912, Pid 28253]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 571 2025-09-11 Thu 04:17:13:124.184 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484345321216, Pid 28253]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 571 2025-09-11 Thu 04:17:13:163.090 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484341122816, Pid 28253]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 571 2025-09-11 Thu 04:17:13:176.539 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484345321216, Pid 28253]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 571 2025-09-11 Thu 04:17:13:204.090 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484315932416, Pid 28253]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 571 2025-09-11 Thu 04:17:13:204.192 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484315932416, Pid 28253]: IFMap [SYS_WARN]: ConfigurationMalformedReferenceWarningLog: bgp_peering No entry in metadata map bgp_router ae44b01e-9ba6-46ee-b793-23f3d46999ec controller/src/ifmap/client/config_json_parser.cc 223 2025-09-11 Thu 04:17:13:221.249 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484315932416, Pid 28253]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 571 2025-09-11 Thu 04:17:13:221.371 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484315932416, Pid 28253]: IFMap [SYS_WARN]: ConfigurationMalformedReferenceWarningLog: bgp_peering No entry in metadata map bgp_router d9886334-d660-45c3-97f9-3f7c9c1d2a94 controller/src/ifmap/client/config_json_parser.cc 223 2025-09-11 Thu 04:17:13:325.700 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484315932416, Pid 28253]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 571 2025-09-11 Thu 04:17:13:325.798 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484315932416, Pid 28253]: IFMap [SYS_WARN]: ConfigurationMalformedReferenceWarningLog: bgp_peering No entry in metadata map bgp_router d9886334-d660-45c3-97f9-3f7c9c1d2a94 controller/src/ifmap/client/config_json_parser.cc 223 2025-09-11 Thu 04:17:13:326.618 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484324329216, Pid 28253]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 571 2025-09-11 Thu 04:17:13:326.706 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484324329216, Pid 28253]: IFMap [SYS_WARN]: ConfigurationMalformedReferenceWarningLog: bgp_peering No entry in metadata map bgp_router 69d58baa-ec2b-4f35-bbd7-4ec0ee893627 controller/src/ifmap/client/config_json_parser.cc 223 2025-09-11 Thu 04:17:13:405.328 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484425504512, Pid 28253]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 571 2025-09-11 Thu 04:17:13:405.421 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484425504512, Pid 28253]: IFMap [SYS_WARN]: ConfigurationMalformedReferenceWarningLog: bgp_peering No entry in metadata map bgp_router ae44b01e-9ba6-46ee-b793-23f3d46999ec controller/src/ifmap/client/config_json_parser.cc 223 2025-09-11 Thu 04:17:13:412.857 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484324329216, Pid 28253]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 571 2025-09-11 Thu 04:17:13:412.986 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484324329216, Pid 28253]: IFMap [SYS_WARN]: ConfigurationMalformedReferenceWarningLog: bgp_peering No entry in metadata map bgp_router 69d58baa-ec2b-4f35-bbd7-4ec0ee893627 controller/src/ifmap/client/config_json_parser.cc 223 2025-09-11 Thu 04:17:13:451.014 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484429702912, Pid 28253]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 571 2025-09-11 Thu 04:17:13:468.712 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484429702912, Pid 28253]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 571 2025-09-11 Thu 04:17:13:511.625 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484345321216, Pid 28253]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 571 2025-09-11 Thu 04:17:13:723.442 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484328527616, Pid 28253]: Processing scm::EvIdleHoldTimerExpired in state Idle 2025-09-11 Thu 04:17:13:723.675 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484328527616, Pid 28253]: Connect : Start Connect timer 10.0.0.254:8086 2025-09-11 Thu 04:17:13:723.784 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484328527616, Pid 28253]: Processing scm::EvSandeshSend in state Connect 2025-09-11 Thu 04:17:13:723.813 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484328527616, Pid 28253]: SANDESH: Send FAILED: 1757564233723510 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-4080-3 process_status= [ [ [ module_id = contrail-dns instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.254:8086, ] ] status = Initializing description = Idle : EvIdleHoldTimerExpired -> Connect ], [ type = Database name = Cassandra server_addrs= [ [ (*_iter6) = 10.0.0.254:9041, (*_iter6) = 10.0.0.22:9041, (*_iter6) = 10.0.0.48:9041, ] ] status = Up description = Established Cassandra connection ], [ type = Database name = RabbitMQ server_addrs= [ [ (*_iter6) = 10.0.0.254:5673, (*_iter6) = 10.0.0.22:5673, (*_iter6) = 10.0.0.48:5673, ] ] status = Up description = RabbitMQ connection established ], ] ] flag_infos= [ [ ] ] description = Collector connection down ], ] ] ] 2025-09-11 Thu 04:17:13:723.821 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484328527616, Pid 28253]: Wrong state: Connect for event: EvSandeshSend message: NodeStatusUVE 2025-09-11 Thu 04:17:13:723.830 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484328527616, Pid 28253]: Processing scm::EvSandeshSend in state Connect 2025-09-11 Thu 04:17:13:723.907 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484585629824, Pid 28253]: -(-1) OnSessionEvent TCP Connect Failed 2025-09-11 Thu 04:17:13:723.932 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484585629824, Pid 28253]: -(-1):EvTcpConnectFail 2025-09-11 Thu 04:17:13:723.984 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484328527616, Pid 28253]: SANDESH: Send FAILED: 1757564233723770 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-4080-3:Control:contrail-dns:0 client_info= [ status = Connect successful_connections = 0 pid = 28253 http_port = 8092 start_time = 1757564225184899 collector_name = collector_ip = 10.0.0.254:8086 collector_list= [ [ (*_iter40) = 10.0.0.254:8086, (*_iter40) = 10.0.0.48:8086, (*_iter40) = 10.0.0.22:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ConfigClientMgrDebug [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 181 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigurationMalformedReferenceWarningLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 10 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = DnsOperational [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 14 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapLinkOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 68 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapNodeOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 78 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 12 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 11 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 356, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 23, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-11 Thu 04:17:13:724.002 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484328527616, Pid 28253]: Wrong state: Connect for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-11 Thu 04:17:13:724.012 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484328527616, Pid 28253]: Processing scm::EvTcpConnectFail in state Connect 2025-09-11 Thu 04:17:13:724.019 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484328527616, Pid 28253]: Connect : EvTcpConnectFail 2025-09-11 Thu 04:17:13:724.109 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484328527616, Pid 28253]: TCP [SYS_ERR]: TcpSessionMessageLog: Session -(-1) > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 312 2025-09-11 Thu 04:17:13:724.138 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484328527616, Pid 28253]: Idle 2025-09-11 Thu 04:17:13:724.199 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484328527616, Pid 28253]: Processing scm::EvSandeshSend in state Idle 2025-09-11 Thu 04:17:13:724.356 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484328527616, Pid 28253]: SANDESH: Send FAILED: 1757564233724069 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-4080-3:Control:contrail-dns:0 client_info= [ status = Connect successful_connections = 0 pid = 28253 http_port = 8092 start_time = 1757564225184899 collector_name = collector_ip = 10.0.0.48:8086 collector_list= [ [ (*_iter40) = 10.0.0.254:8086, (*_iter40) = 10.0.0.48:8086, (*_iter40) = 10.0.0.22:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ConfigClientMgrDebug [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 181 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigurationMalformedReferenceWarningLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 10 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = DnsOperational [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 14 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapLinkOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 68 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapNodeOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 78 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 13 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 12 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 356, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 25, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-11 Thu 04:17:13:724.371 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484328527616, Pid 28253]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-11 Thu 04:17:13:724.380 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484328527616, Pid 28253]: Processing scm::EvSandeshSend in state Idle 2025-09-11 Thu 04:17:13:724.399 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484328527616, Pid 28253]: SANDESH: Send FAILED: 1757564233724089 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-4080-3 process_status= [ [ [ module_id = contrail-dns instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], [ type = Database name = Cassandra server_addrs= [ [ (*_iter6) = 10.0.0.254:9041, (*_iter6) = 10.0.0.22:9041, (*_iter6) = 10.0.0.48:9041, ] ] status = Up description = Established Cassandra connection ], [ type = Database name = RabbitMQ server_addrs= [ [ (*_iter6) = 10.0.0.254:5673, (*_iter6) = 10.0.0.22:5673, (*_iter6) = 10.0.0.48:5673, ] ] status = Up description = RabbitMQ connection established ], ] ] flag_infos= [ [ ] ] description = Collector connection down ], ] ] ] 2025-09-11 Thu 04:17:13:724.406 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484328527616, Pid 28253]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-11 Thu 04:17:13:724.413 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484328527616, Pid 28253]: Processing scm::EvTcpDeleteSession in state Idle 2025-09-11 Thu 04:17:13:724.430 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484328527616, Pid 28253]: Processing scm::EvSandeshSend in state Idle 2025-09-11 Thu 04:17:13:724.576 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484328527616, Pid 28253]: SANDESH: Send FAILED: 1757564233724188 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-4080-3:Control:contrail-dns:0 client_info= [ status = Idle successful_connections = 0 pid = 28253 http_port = 8092 start_time = 1757564225184899 collector_name = collector_ip = 10.0.0.48:8086 collector_list= [ [ (*_iter40) = 10.0.0.254:8086, (*_iter40) = 10.0.0.48:8086, (*_iter40) = 10.0.0.22:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ConfigClientMgrDebug [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 181 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigurationMalformedReferenceWarningLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 10 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = DnsOperational [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 14 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapLinkOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 68 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapNodeOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 78 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 13 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 12 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 4 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 357, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 25, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-11 Thu 04:17:13:724.592 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484328527616, Pid 28253]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-11 Thu 04:17:14:485.305 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484353718016, Pid 28253]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 571 2025-09-11 Thu 04:17:14:544.303 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484315932416, Pid 28253]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 571 2025-09-11 Thu 04:17:14:554.628 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484328527616, Pid 28253]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 571 2025-09-11 Thu 04:17:15:364.503 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484315932416, Pid 28253]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 571 2025-09-11 Thu 04:17:21:724.259 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484345321216, Pid 28253]: Processing scm::EvIdleHoldTimerExpired in state Idle 2025-09-11 Thu 04:17:21:724.495 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484345321216, Pid 28253]: Connect : Start Connect timer 10.0.0.48:8086 2025-09-11 Thu 04:17:21:724.627 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484345321216, Pid 28253]: Processing scm::EvSandeshSend in state Connect 2025-09-11 Thu 04:17:21:724.668 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484345321216, Pid 28253]: SANDESH: Send FAILED: 1757564241724342 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-4080-3 process_status= [ [ [ module_id = contrail-dns instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.48:8086, ] ] status = Initializing description = Idle : EvIdleHoldTimerExpired -> Connect ], [ type = Database name = Cassandra server_addrs= [ [ (*_iter6) = 10.0.0.254:9041, (*_iter6) = 10.0.0.22:9041, (*_iter6) = 10.0.0.48:9041, ] ] status = Up description = Established Cassandra connection ], [ type = Database name = RabbitMQ server_addrs= [ [ (*_iter6) = 10.0.0.254:5673, (*_iter6) = 10.0.0.22:5673, (*_iter6) = 10.0.0.48:5673, ] ] status = Up description = RabbitMQ connection established ], ] ] flag_infos= [ [ ] ] description = Collector connection down ], ] ] ] 2025-09-11 Thu 04:17:21:724.677 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484345321216, Pid 28253]: Wrong state: Connect for event: EvSandeshSend message: NodeStatusUVE 2025-09-11 Thu 04:17:21:724.685 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484345321216, Pid 28253]: Processing scm::EvSandeshSend in state Connect 2025-09-11 Thu 04:17:21:724.832 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484345321216, Pid 28253]: SANDESH: Send FAILED: 1757564241724613 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-4080-3:Control:contrail-dns:0 client_info= [ status = Connect successful_connections = 0 pid = 28253 http_port = 8092 start_time = 1757564225184899 collector_name = collector_ip = 10.0.0.48:8086 collector_list= [ [ (*_iter40) = 10.0.0.254:8086, (*_iter40) = 10.0.0.48:8086, (*_iter40) = 10.0.0.22:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ConfigClientMgrDebug [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 185 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigurationMalformedReferenceWarningLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 10 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = DnsOperational [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 14 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapLinkOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 68 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapNodeOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 78 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 14 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 14 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 4 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 361, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 28, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-11 Thu 04:17:21:724.849 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484345321216, Pid 28253]: Wrong state: Connect for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-11 Thu 04:17:21:724.875 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484585629824, Pid 28253]: -(-1) OnSessionEvent TCP Connect Failed 2025-09-11 Thu 04:17:21:724.884 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484585629824, Pid 28253]: -(-1):EvTcpConnectFail 2025-09-11 Thu 04:17:21:724.937 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484315932416, Pid 28253]: Processing scm::EvTcpConnectFail in state Connect 2025-09-11 Thu 04:17:21:724.969 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484315932416, Pid 28253]: Connect : EvTcpConnectFail 2025-09-11 Thu 04:17:21:725.105 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484315932416, Pid 28253]: TCP [SYS_ERR]: TcpSessionMessageLog: Session -(-1) > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 312 2025-09-11 Thu 04:17:21:725.140 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484315932416, Pid 28253]: Idle 2025-09-11 Thu 04:17:21:725.215 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484315932416, Pid 28253]: Processing scm::EvSandeshSend in state Idle 2025-09-11 Thu 04:17:21:725.371 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484315932416, Pid 28253]: SANDESH: Send FAILED: 1757564241725054 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-4080-3:Control:contrail-dns:0 client_info= [ status = Connect successful_connections = 0 pid = 28253 http_port = 8092 start_time = 1757564225184899 collector_name = collector_ip = 10.0.0.22:8086 collector_list= [ [ (*_iter40) = 10.0.0.254:8086, (*_iter40) = 10.0.0.48:8086, (*_iter40) = 10.0.0.22:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ConfigClientMgrDebug [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 185 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigurationMalformedReferenceWarningLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 10 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = DnsOperational [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 14 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapLinkOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 68 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapNodeOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 78 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 15 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 15 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 4 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 361, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 30, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-11 Thu 04:17:21:725.392 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484315932416, Pid 28253]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-11 Thu 04:17:21:725.402 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484315932416, Pid 28253]: Processing scm::EvSandeshSend in state Idle 2025-09-11 Thu 04:17:21:725.422 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484315932416, Pid 28253]: SANDESH: Send FAILED: 1757564241725081 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-4080-3 process_status= [ [ [ module_id = contrail-dns instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], [ type = Database name = Cassandra server_addrs= [ [ (*_iter6) = 10.0.0.254:9041, (*_iter6) = 10.0.0.22:9041, (*_iter6) = 10.0.0.48:9041, ] ] status = Up description = Established Cassandra connection ], [ type = Database name = RabbitMQ server_addrs= [ [ (*_iter6) = 10.0.0.254:5673, (*_iter6) = 10.0.0.22:5673, (*_iter6) = 10.0.0.48:5673, ] ] status = Up description = RabbitMQ connection established ], ] ] flag_infos= [ [ ] ] description = Collector connection down ], ] ] ] 2025-09-11 Thu 04:17:21:725.429 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484315932416, Pid 28253]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-11 Thu 04:17:21:725.437 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484315932416, Pid 28253]: Processing scm::EvTcpDeleteSession in state Idle 2025-09-11 Thu 04:17:21:725.454 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484315932416, Pid 28253]: Processing scm::EvSandeshSend in state Idle 2025-09-11 Thu 04:17:21:725.589 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484315932416, Pid 28253]: SANDESH: Send FAILED: 1757564241725194 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-4080-3:Control:contrail-dns:0 client_info= [ status = Idle successful_connections = 0 pid = 28253 http_port = 8092 start_time = 1757564225184899 collector_name = collector_ip = 10.0.0.22:8086 collector_list= [ [ (*_iter40) = 10.0.0.254:8086, (*_iter40) = 10.0.0.48:8086, (*_iter40) = 10.0.0.22:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ConfigClientMgrDebug [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 185 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigurationMalformedReferenceWarningLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 10 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = DnsOperational [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 14 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapLinkOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 68 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapNodeOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 78 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 15 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 15 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 5 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 362, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 30, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-11 Thu 04:17:21:725.603 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484315932416, Pid 28253]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-11 Thu 04:17:35:185.378 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484324329216, Pid 28253]: Processing scm::EvSandeshSend in state Idle 2025-09-11 Thu 04:17:35:185.493 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484324329216, Pid 28253]: SANDESH: Send FAILED: 1757564255185194 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-4080-3:Control:contrail-dns:0 sm_queue_count = 0 sm_stats= [ ev_stats= [ [ [ event = scm::EvCollectorUpdate enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvIdleHoldTimerExpired enqueues = 4 dequeues = 4 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvSandeshSend enqueues = 33 dequeues = 33 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvStart enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpConnectFail enqueues = 5 dequeues = 5 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpDeleteSession enqueues = 5 dequeues = 5 enqueue_fails = 0 dequeue_fails = 0 ], [ event = enqueues = 49 dequeues = 49 enqueue_fails = 0 dequeue_fails = 0 ], ] ] state = Idle state_since = 1757564241725139 last_state = Idle last_event = scm::EvSandeshSend last_event_at = 1757564241725452 ] max_sm_queue_count = 5 ] 2025-09-11 Thu 04:17:35:185.514 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484324329216, Pid 28253]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-11 Thu 04:17:35:185.525 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484324329216, Pid 28253]: Processing scm::EvSandeshSend in state Idle 2025-09-11 Thu 04:17:35:185.709 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484324329216, Pid 28253]: SANDESH: Send FAILED: 1757564255185317 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-4080-3:Control:contrail-dns:0 client_info= [ status = Idle successful_connections = 0 pid = 28253 http_port = 8092 start_time = 1757564225184899 collector_name = collector_ip = 10.0.0.22:8086 collector_list= [ [ (*_iter40) = 10.0.0.254:8086, (*_iter40) = 10.0.0.48:8086, (*_iter40) = 10.0.0.22:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ConfigClientMgrDebug [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 185 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigurationMalformedReferenceWarningLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 10 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = DnsOperational [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 14 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapLinkOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 68 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapNodeOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 78 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 16 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 17 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 5 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 362, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 33, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-11 Thu 04:17:35:185.731 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484324329216, Pid 28253]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-11 Thu 04:17:37:725.248 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484332726016, Pid 28253]: Processing scm::EvIdleHoldTimerExpired in state Idle 2025-09-11 Thu 04:17:37:725.490 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484332726016, Pid 28253]: Connect : Start Connect timer 10.0.0.22:8086 2025-09-11 Thu 04:17:37:725.602 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484332726016, Pid 28253]: Processing scm::EvSandeshSend in state Connect 2025-09-11 Thu 04:17:37:725.631 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484332726016, Pid 28253]: SANDESH: Send FAILED: 1757564257725321 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-4080-3 process_status= [ [ [ module_id = contrail-dns instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.22:8086, ] ] status = Initializing description = Idle : EvIdleHoldTimerExpired -> Connect ], [ type = Database name = Cassandra server_addrs= [ [ (*_iter6) = 10.0.0.254:9041, (*_iter6) = 10.0.0.22:9041, (*_iter6) = 10.0.0.48:9041, ] ] status = Up description = Established Cassandra connection ], [ type = Database name = RabbitMQ server_addrs= [ [ (*_iter6) = 10.0.0.254:5673, (*_iter6) = 10.0.0.22:5673, (*_iter6) = 10.0.0.48:5673, ] ] status = Up description = RabbitMQ connection established ], ] ] flag_infos= [ [ ] ] description = Collector connection down ], ] ] ] 2025-09-11 Thu 04:17:37:725.669 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484332726016, Pid 28253]: Wrong state: Connect for event: EvSandeshSend message: NodeStatusUVE 2025-09-11 Thu 04:17:37:725.679 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484332726016, Pid 28253]: Processing scm::EvSandeshSend in state Connect 2025-09-11 Thu 04:17:37:725.851 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484332726016, Pid 28253]: SANDESH: Send FAILED: 1757564257725584 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-4080-3:Control:contrail-dns:0 client_info= [ status = Connect successful_connections = 0 pid = 28253 http_port = 8092 start_time = 1757564225184899 collector_name = collector_ip = 10.0.0.22:8086 collector_list= [ [ (*_iter40) = 10.0.0.254:8086, (*_iter40) = 10.0.0.48:8086, (*_iter40) = 10.0.0.22:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ConfigClientMgrDebug [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 185 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigurationMalformedReferenceWarningLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 10 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = DnsOperational [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 14 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapLinkOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 68 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapNodeOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 78 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 16 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 19 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 5 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 362, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 35, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-11 Thu 04:17:37:725.868 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484332726016, Pid 28253]: Wrong state: Connect for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-11 Thu 04:17:37:725.900 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484585629824, Pid 28253]: -(-1) OnSessionEvent TCP Connect Failed 2025-09-11 Thu 04:17:37:725.908 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484585629824, Pid 28253]: -(-1):EvTcpConnectFail 2025-09-11 Thu 04:17:37:725.937 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484332726016, Pid 28253]: Processing scm::EvTcpConnectFail in state Connect 2025-09-11 Thu 04:17:37:725.944 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484332726016, Pid 28253]: Connect : EvTcpConnectFail 2025-09-11 Thu 04:17:37:726.036 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484332726016, Pid 28253]: TCP [SYS_ERR]: TcpSessionMessageLog: Session -(-1) > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 312 2025-09-11 Thu 04:17:37:726.067 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484332726016, Pid 28253]: Idle 2025-09-11 Thu 04:17:37:726.145 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484332726016, Pid 28253]: Processing scm::EvSandeshSend in state Idle 2025-09-11 Thu 04:17:37:726.289 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484332726016, Pid 28253]: SANDESH: Send FAILED: 1757564257726000 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-4080-3:Control:contrail-dns:0 client_info= [ status = Connect successful_connections = 0 pid = 28253 http_port = 8092 start_time = 1757564225184899 collector_name = collector_ip = 10.0.0.254:8086 collector_list= [ [ (*_iter40) = 10.0.0.254:8086, (*_iter40) = 10.0.0.48:8086, (*_iter40) = 10.0.0.22:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ConfigClientMgrDebug [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 185 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigurationMalformedReferenceWarningLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 10 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = DnsOperational [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 14 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapLinkOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 68 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapNodeOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 78 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 17 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 20 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 5 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 362, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 37, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-11 Thu 04:17:37:726.306 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484332726016, Pid 28253]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-11 Thu 04:17:37:726.316 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484332726016, Pid 28253]: Processing scm::EvSandeshSend in state Idle 2025-09-11 Thu 04:17:37:726.350 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484332726016, Pid 28253]: SANDESH: Send FAILED: 1757564257726020 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-4080-3 process_status= [ [ [ module_id = contrail-dns instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], [ type = Database name = Cassandra server_addrs= [ [ (*_iter6) = 10.0.0.254:9041, (*_iter6) = 10.0.0.22:9041, (*_iter6) = 10.0.0.48:9041, ] ] status = Up description = Established Cassandra connection ], [ type = Database name = RabbitMQ server_addrs= [ [ (*_iter6) = 10.0.0.254:5673, (*_iter6) = 10.0.0.22:5673, (*_iter6) = 10.0.0.48:5673, ] ] status = Up description = RabbitMQ connection established ], ] ] flag_infos= [ [ ] ] description = Collector connection down ], ] ] ] 2025-09-11 Thu 04:17:37:726.357 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484332726016, Pid 28253]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-11 Thu 04:17:37:726.365 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484332726016, Pid 28253]: Processing scm::EvTcpDeleteSession in state Idle 2025-09-11 Thu 04:17:37:726.384 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484332726016, Pid 28253]: Processing scm::EvSandeshSend in state Idle 2025-09-11 Thu 04:17:37:726.519 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484332726016, Pid 28253]: SANDESH: Send FAILED: 1757564257726135 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-4080-3:Control:contrail-dns:0 client_info= [ status = Idle successful_connections = 0 pid = 28253 http_port = 8092 start_time = 1757564225184899 collector_name = collector_ip = 10.0.0.254:8086 collector_list= [ [ (*_iter40) = 10.0.0.254:8086, (*_iter40) = 10.0.0.48:8086, (*_iter40) = 10.0.0.22:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ConfigClientMgrDebug [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 185 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigurationMalformedReferenceWarningLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 10 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = DnsOperational [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 14 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapLinkOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 68 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapNodeOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 78 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 17 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 20 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 6 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 363, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 37, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-11 Thu 04:17:37:726.535 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484332726016, Pid 28253]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-11 Thu 04:17:42:513.112 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484429702912, Pid 28253]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 571 2025-09-11 Thu 04:17:42:513.261 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484429702912, Pid 28253]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating database-node:default-global-system-config:cn-jenkins-deploy-platform-ansible-os-4080-2. controller/src/ifmap/ifmap_server_table.cc 95 2025-09-11 Thu 04:17:42:513.296 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484429702912, Pid 28253]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-database-node controller/src/ifmap/ifmap_server_table.cc 130 2025-09-11 Thu 04:17:42:514.680 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484349519616, Pid 28253]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 571 2025-09-11 Thu 04:17:42:514.805 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484315932416, Pid 28253]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating database-node:default-global-system-config:cn-jenkins-deploy-platform-ansible-os-4080-3. controller/src/ifmap/ifmap_server_table.cc 95 2025-09-11 Thu 04:17:42:514.849 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484315932416, Pid 28253]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-database-node controller/src/ifmap/ifmap_server_table.cc 130 2025-09-11 Thu 04:17:42:557.974 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484315932416, Pid 28253]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 571 2025-09-11 Thu 04:17:42:558.100 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484315932416, Pid 28253]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating database-node:default-global-system-config:cn-jenkins-deploy-platform-ansible-os-4080-1. controller/src/ifmap/ifmap_server_table.cc 95 2025-09-11 Thu 04:17:42:558.138 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484315932416, Pid 28253]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-database-node controller/src/ifmap/ifmap_server_table.cc 130 2025-09-11 Thu 04:18:05:185.584 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484349519616, Pid 28253]: Processing scm::EvSandeshSend in state Idle 2025-09-11 Thu 04:18:05:185.724 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484349519616, Pid 28253]: SANDESH: Send FAILED: 1757564285185434 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-4080-3:Control:contrail-dns:0 sm_queue_count = 0 sm_stats= [ ev_stats= [ [ [ event = scm::EvCollectorUpdate enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvIdleHoldTimerExpired enqueues = 5 dequeues = 5 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvSandeshSend enqueues = 40 dequeues = 40 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvStart enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpConnectFail enqueues = 6 dequeues = 6 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpDeleteSession enqueues = 6 dequeues = 6 enqueue_fails = 0 dequeue_fails = 0 ], [ event = enqueues = 59 dequeues = 59 enqueue_fails = 0 dequeue_fails = 0 ], ] ] state = Idle state_since = 1757564257726066 last_state = Idle last_event = scm::EvSandeshSend last_event_at = 1757564257726383 ] max_sm_queue_count = 5 ] 2025-09-11 Thu 04:18:05:185.756 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484349519616, Pid 28253]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-11 Thu 04:18:05:185.771 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484349519616, Pid 28253]: Processing scm::EvSandeshSend in state Idle 2025-09-11 Thu 04:18:05:186.035 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484349519616, Pid 28253]: SANDESH: Send FAILED: 1757564285185541 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-4080-3:Control:contrail-dns:0 client_info= [ status = Idle successful_connections = 0 pid = 28253 http_port = 8092 start_time = 1757564225184899 collector_name = collector_ip = 10.0.0.254:8086 collector_list= [ [ (*_iter40) = 10.0.0.254:8086, (*_iter40) = 10.0.0.48:8086, (*_iter40) = 10.0.0.22:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ConfigClientMgrDebug [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 188 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigurationMalformedReferenceWarningLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 10 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = DnsOperational [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 14 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapLinkOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 71 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapNodeOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 81 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 18 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 22 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 6 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 372, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 40, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-11 Thu 04:18:05:186.056 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484349519616, Pid 28253]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-11 Thu 04:18:05:579.203 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484341122816, Pid 28253]: Processing scm::EvSandeshSend in state Idle 2025-09-11 Thu 04:18:05:579.271 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484341122816, Pid 28253]: SANDESH: Send FAILED: 1757564285579102 [SYS_NOTICE]: UveDnsInfo: data= [ name = cn-jenkins-deploy-platform-ansible-os-4080-3 start_time = 1757564225378784 build_info = {"build-info":[{"build-version":"v0+nightly.r24.1.centos","build-time":"2025-09-11 00:09:15.426696","build-user":"root","build-hostname":"jenkins-build-prepare-sandbox-10925-1.novalocal","build-id":"v0+nightly.r24.1.centos-nightly_r24.1_centos.el7@contrail","build-number":"unknown"}]} collector = 10.0.0.254:8086 self_ip_list= [ [ (*_iter12) = 10.20.0.14, ] ] ] 2025-09-11 Thu 04:18:05:579.289 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484341122816, Pid 28253]: Wrong state: Idle for event: EvSandeshSend message: UveDnsInfo 2025-09-11 Thu 04:18:07:726.199 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484328527616, Pid 28253]: Processing scm::EvIdleHoldTimerExpired in state Idle 2025-09-11 Thu 04:18:07:726.483 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484328527616, Pid 28253]: Connect : Start Connect timer 10.0.0.254:8086 2025-09-11 Thu 04:18:07:726.665 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484328527616, Pid 28253]: Processing scm::EvSandeshSend in state Connect 2025-09-11 Thu 04:18:07:727.523 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484328527616, Pid 28253]: SANDESH: Send FAILED: 1757564287726299 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-4080-3 process_status= [ [ [ module_id = contrail-dns instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.254:8086, ] ] status = Initializing description = Idle : EvIdleHoldTimerExpired -> Connect ], [ type = Database name = Cassandra server_addrs= [ [ (*_iter6) = 10.0.0.254:9041, (*_iter6) = 10.0.0.22:9041, (*_iter6) = 10.0.0.48:9041, ] ] status = Up description = Established Cassandra connection ], [ type = Database name = RabbitMQ server_addrs= [ [ (*_iter6) = 10.0.0.254:5673, (*_iter6) = 10.0.0.22:5673, (*_iter6) = 10.0.0.48:5673, ] ] status = Up description = RabbitMQ connection established ], ] ] flag_infos= [ [ ] ] description = Collector connection down ], ] ] ] 2025-09-11 Thu 04:18:07:727.535 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484328527616, Pid 28253]: Wrong state: Connect for event: EvSandeshSend message: NodeStatusUVE 2025-09-11 Thu 04:18:07:727.544 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484328527616, Pid 28253]: Processing scm::EvSandeshSend in state Connect 2025-09-11 Thu 04:18:07:727.711 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484328527616, Pid 28253]: SANDESH: Send FAILED: 1757564287726648 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-4080-3:Control:contrail-dns:0 client_info= [ status = Connect successful_connections = 0 pid = 28253 http_port = 8092 start_time = 1757564225184899 collector_name = collector_ip = 10.0.0.254:8086 collector_list= [ [ (*_iter40) = 10.0.0.254:8086, (*_iter40) = 10.0.0.48:8086, (*_iter40) = 10.0.0.22:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ConfigClientMgrDebug [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 188 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigurationMalformedReferenceWarningLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 10 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = DnsOperational [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 14 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapLinkOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 71 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapNodeOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 81 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 18 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 24 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 6 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveDnsInfo [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 372, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 43, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-11 Thu 04:18:07:727.731 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484328527616, Pid 28253]: Wrong state: Connect for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-11 Thu 04:18:07:726.858 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484585629824, Pid 28253]: -(-1) OnSessionEvent TCP Connect Failed 2025-09-11 Thu 04:18:07:727.750 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484585629824, Pid 28253]: -(-1):EvTcpConnectFail 2025-09-11 Thu 04:18:07:727.789 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484349519616, Pid 28253]: Processing scm::EvTcpConnectFail in state Connect 2025-09-11 Thu 04:18:07:727.800 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484349519616, Pid 28253]: Connect : EvTcpConnectFail 2025-09-11 Thu 04:18:07:727.924 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484349519616, Pid 28253]: TCP [SYS_ERR]: TcpSessionMessageLog: Session -(-1) > Shutdown failed due to error: Transport endpoint is not connected src/contrail-common/io/tcp_session.cc 312 2025-09-11 Thu 04:18:07:727.959 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484349519616, Pid 28253]: Idle 2025-09-11 Thu 04:18:07:728.016 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484349519616, Pid 28253]: Processing scm::EvSandeshSend in state Idle 2025-09-11 Thu 04:18:07:729.323 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484349519616, Pid 28253]: SANDESH: Send FAILED: 1757564287727882 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-4080-3:Control:contrail-dns:0 client_info= [ status = Connect successful_connections = 0 pid = 28253 http_port = 8092 start_time = 1757564225184899 collector_name = collector_ip = 10.0.0.48:8086 collector_list= [ [ (*_iter40) = 10.0.0.254:8086, (*_iter40) = 10.0.0.48:8086, (*_iter40) = 10.0.0.22:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ConfigClientMgrDebug [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 188 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigurationMalformedReferenceWarningLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 10 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = DnsOperational [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 14 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapLinkOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 71 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapNodeOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 81 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 19 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 25 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 6 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveDnsInfo [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 372, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 45, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-11 Thu 04:18:07:730.225 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484349519616, Pid 28253]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-11 Thu 04:18:07:730.265 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484349519616, Pid 28253]: Processing scm::EvSandeshSend in state Idle 2025-09-11 Thu 04:18:07:730.295 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484349519616, Pid 28253]: SANDESH: Send FAILED: 1757564287727909 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-4080-3 process_status= [ [ [ module_id = contrail-dns instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvTcpConnectFail ], [ type = Database name = Cassandra server_addrs= [ [ (*_iter6) = 10.0.0.254:9041, (*_iter6) = 10.0.0.22:9041, (*_iter6) = 10.0.0.48:9041, ] ] status = Up description = Established Cassandra connection ], [ type = Database name = RabbitMQ server_addrs= [ [ (*_iter6) = 10.0.0.254:5673, (*_iter6) = 10.0.0.22:5673, (*_iter6) = 10.0.0.48:5673, ] ] status = Up description = RabbitMQ connection established ], ] ] flag_infos= [ [ ] ] description = Collector connection down ], ] ] ] 2025-09-11 Thu 04:18:07:730.303 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484349519616, Pid 28253]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-09-11 Thu 04:18:07:730.312 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484349519616, Pid 28253]: Processing scm::EvTcpDeleteSession in state Idle 2025-09-11 Thu 04:18:07:730.339 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484349519616, Pid 28253]: Processing scm::EvSandeshSend in state Idle 2025-09-11 Thu 04:18:07:730.506 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484349519616, Pid 28253]: SANDESH: Send FAILED: 1757564287728007 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-4080-3:Control:contrail-dns:0 client_info= [ status = Idle successful_connections = 0 pid = 28253 http_port = 8092 start_time = 1757564225184899 collector_name = collector_ip = 10.0.0.48:8086 collector_list= [ [ (*_iter40) = 10.0.0.254:8086, (*_iter40) = 10.0.0.48:8086, (*_iter40) = 10.0.0.22:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ConfigClientMgrDebug [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 188 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigurationMalformedReferenceWarningLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 10 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = DnsOperational [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 14 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapLinkOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 71 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapNodeOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 81 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 19 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 25 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 7 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveDnsInfo [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 373, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 45, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-11 Thu 04:18:07:730.521 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484349519616, Pid 28253]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-11 Thu 04:18:10:051.453 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484349519616, Pid 28253]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 571 2025-09-11 Thu 04:18:10:051.637 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484425504512, Pid 28253]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating analytics-node:default-global-system-config:cn-jenkins-deploy-platform-ansible-os-4080-3. controller/src/ifmap/ifmap_server_table.cc 95 2025-09-11 Thu 04:18:10:051.712 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484425504512, Pid 28253]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-analytics-node controller/src/ifmap/ifmap_server_table.cc 130 2025-09-11 Thu 04:18:10:082.418 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484315932416, Pid 28253]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 571 2025-09-11 Thu 04:18:10:082.590 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484349519616, Pid 28253]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating analytics-node:default-global-system-config:cn-jenkins-deploy-platform-ansible-os-4080-2. controller/src/ifmap/ifmap_server_table.cc 95 2025-09-11 Thu 04:18:10:082.644 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484349519616, Pid 28253]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-analytics-node controller/src/ifmap/ifmap_server_table.cc 130 2025-09-11 Thu 04:18:10:183.150 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484328527616, Pid 28253]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 571 2025-09-11 Thu 04:18:10:183.295 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484324329216, Pid 28253]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating analytics-node:default-global-system-config:cn-jenkins-deploy-platform-ansible-os-4080-1. controller/src/ifmap/ifmap_server_table.cc 95 2025-09-11 Thu 04:18:10:183.362 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484324329216, Pid 28253]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-analytics-node controller/src/ifmap/ifmap_server_table.cc 130 2025-09-11 Thu 04:18:24:767.261 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484311734016, Pid 28253]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 571 2025-09-11 Thu 04:18:24:767.429 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484320130816, Pid 28253]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating analytics-alarm-node:default-global-system-config:cn-jenkins-deploy-platform-ansible-os-4080-1. controller/src/ifmap/ifmap_server_table.cc 95 2025-09-11 Thu 04:18:24:767.486 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484320130816, Pid 28253]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-analytics-alarm-node controller/src/ifmap/ifmap_server_table.cc 130 2025-09-11 Thu 04:18:25:207.545 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484315932416, Pid 28253]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 571 2025-09-11 Thu 04:18:25:207.720 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484345321216, Pid 28253]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating analytics-alarm-node:default-global-system-config:cn-jenkins-deploy-platform-ansible-os-4080-2. controller/src/ifmap/ifmap_server_table.cc 95 2025-09-11 Thu 04:18:25:207.778 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484345321216, Pid 28253]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-analytics-alarm-node controller/src/ifmap/ifmap_server_table.cc 130 2025-09-11 Thu 04:18:25:243.813 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484315932416, Pid 28253]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 571 2025-09-11 Thu 04:18:25:245.369 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484315932416, Pid 28253]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating analytics-alarm-node:default-global-system-config:cn-jenkins-deploy-platform-ansible-os-4080-3. controller/src/ifmap/ifmap_server_table.cc 95 2025-09-11 Thu 04:18:25:245.404 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484315932416, Pid 28253]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-analytics-alarm-node controller/src/ifmap/ifmap_server_table.cc 130 2025-09-11 Thu 04:18:34:583.563 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484332726016, Pid 28253]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 571 2025-09-11 Thu 04:18:34:583.719 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484332726016, Pid 28253]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating analytics-snmp-node:default-global-system-config:cn-jenkins-deploy-platform-ansible-os-4080-3. controller/src/ifmap/ifmap_server_table.cc 95 2025-09-11 Thu 04:18:34:583.753 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484332726016, Pid 28253]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-analytics-snmp-node controller/src/ifmap/ifmap_server_table.cc 130 2025-09-11 Thu 04:18:34:629.568 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484311734016, Pid 28253]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 571 2025-09-11 Thu 04:18:34:629.740 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484345321216, Pid 28253]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating analytics-snmp-node:default-global-system-config:cn-jenkins-deploy-platform-ansible-os-4080-2. controller/src/ifmap/ifmap_server_table.cc 95 2025-09-11 Thu 04:18:34:629.829 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484345321216, Pid 28253]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-analytics-snmp-node controller/src/ifmap/ifmap_server_table.cc 130 2025-09-11 Thu 04:18:34:652.651 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484353718016, Pid 28253]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 571 2025-09-11 Thu 04:18:34:652.806 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484353718016, Pid 28253]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating analytics-snmp-node:default-global-system-config:cn-jenkins-deploy-platform-ansible-os-4080-1. controller/src/ifmap/ifmap_server_table.cc 95 2025-09-11 Thu 04:18:34:652.839 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484353718016, Pid 28253]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-analytics-snmp-node controller/src/ifmap/ifmap_server_table.cc 130 2025-09-11 Thu 04:18:35:185.803 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484345321216, Pid 28253]: Processing scm::EvSandeshSend in state Idle 2025-09-11 Thu 04:18:35:185.885 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484345321216, Pid 28253]: SANDESH: Send FAILED: 1757564315185653 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-4080-3:Control:contrail-dns:0 sm_queue_count = 0 sm_stats= [ ev_stats= [ [ [ event = scm::EvCollectorUpdate enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvIdleHoldTimerExpired enqueues = 6 dequeues = 6 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvSandeshSend enqueues = 48 dequeues = 48 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvStart enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpConnectFail enqueues = 7 dequeues = 7 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpDeleteSession enqueues = 7 dequeues = 7 enqueue_fails = 0 dequeue_fails = 0 ], [ event = enqueues = 70 dequeues = 70 enqueue_fails = 0 dequeue_fails = 0 ], ] ] state = Idle state_since = 1757564287727958 last_state = Idle last_event = scm::EvSandeshSend last_event_at = 1757564287730338 ] max_sm_queue_count = 5 ] 2025-09-11 Thu 04:18:35:185.898 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484345321216, Pid 28253]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-11 Thu 04:18:35:185.908 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484345321216, Pid 28253]: Processing scm::EvSandeshSend in state Idle 2025-09-11 Thu 04:18:35:186.077 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484345321216, Pid 28253]: SANDESH: Send FAILED: 1757564315185759 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-4080-3:Control:contrail-dns:0 client_info= [ status = Idle successful_connections = 0 pid = 28253 http_port = 8092 start_time = 1757564225184899 collector_name = collector_ip = 10.0.0.48:8086 collector_list= [ [ (*_iter40) = 10.0.0.254:8086, (*_iter40) = 10.0.0.48:8086, (*_iter40) = 10.0.0.22:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ConfigClientMgrDebug [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 197 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigurationMalformedReferenceWarningLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 10 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = DnsOperational [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 14 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapLinkOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 80 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapNodeOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 90 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 20 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 27 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 7 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveDnsInfo [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 400, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 48, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-11 Thu 04:18:35:186.095 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484345321216, Pid 28253]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-11 Thu 04:18:37:728.086 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484345321216, Pid 28253]: Processing scm::EvIdleHoldTimerExpired in state Idle 2025-09-11 Thu 04:18:37:728.349 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484345321216, Pid 28253]: Connect : Start Connect timer 10.0.0.48:8086 2025-09-11 Thu 04:18:37:728.475 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484345321216, Pid 28253]: Processing scm::EvSandeshSend in state Connect 2025-09-11 Thu 04:18:37:728.506 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484345321216, Pid 28253]: SANDESH: Send FAILED: 1757564317728167 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-4080-3 process_status= [ [ [ module_id = contrail-dns instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.48:8086, ] ] status = Initializing description = Idle : EvIdleHoldTimerExpired -> Connect ], [ type = Database name = Cassandra server_addrs= [ [ (*_iter6) = 10.0.0.254:9041, (*_iter6) = 10.0.0.22:9041, (*_iter6) = 10.0.0.48:9041, ] ] status = Up description = Established Cassandra connection ], [ type = Database name = RabbitMQ server_addrs= [ [ (*_iter6) = 10.0.0.254:5673, (*_iter6) = 10.0.0.22:5673, (*_iter6) = 10.0.0.48:5673, ] ] status = Up description = RabbitMQ connection established ], ] ] flag_infos= [ [ ] ] description = Collector connection down ], ] ] ] 2025-09-11 Thu 04:18:37:728.526 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484345321216, Pid 28253]: Wrong state: Connect for event: EvSandeshSend message: NodeStatusUVE 2025-09-11 Thu 04:18:37:728.535 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484345321216, Pid 28253]: Processing scm::EvSandeshSend in state Connect 2025-09-11 Thu 04:18:37:728.722 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484345321216, Pid 28253]: SANDESH: Send FAILED: 1757564317728449 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-4080-3:Control:contrail-dns:0 client_info= [ status = Connect successful_connections = 0 pid = 28253 http_port = 8092 start_time = 1757564225184899 collector_name = collector_ip = 10.0.0.48:8086 collector_list= [ [ (*_iter40) = 10.0.0.254:8086, (*_iter40) = 10.0.0.48:8086, (*_iter40) = 10.0.0.22:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ConfigClientMgrDebug [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 197 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigurationMalformedReferenceWarningLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 10 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = DnsOperational [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 14 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapLinkOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 80 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapNodeOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 90 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 20 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 29 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 7 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveDnsInfo [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 400, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 50, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-11 Thu 04:18:37:728.741 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484345321216, Pid 28253]: Wrong state: Connect for event: EvSandeshSend message: SandeshModuleClientTrace 2025-09-11 Thu 04:18:37:731.195 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484585629824, Pid 28253]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.0.0.48:54432::10.0.0.48:8086(-1) < Active session connection complete src/contrail-common/io/tcp_session.cc 280 2025-09-11 Thu 04:18:37:731.225 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484585629824, Pid 28253]: 10.0.0.48:54432::10.0.0.48:8086(-1) OnSessionEvent TCP Connected 2025-09-11 Thu 04:18:37:731.233 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484585629824, Pid 28253]: 10.0.0.48:54432::10.0.0.48:8086(-1):EvTcpConnected 2025-09-11 Thu 04:18:37:731.303 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484345321216, Pid 28253]: Processing scm::EvTcpConnected in state Connect 2025-09-11 Thu 04:18:37:731.340 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484345321216, Pid 28253]: Connect : EvTcpConnected : Cancelling Connect timer 2025-09-11 Thu 04:18:37:731.384 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484345321216, Pid 28253]: ClientInit 2025-09-11 Thu 04:18:37:731.412 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484345321216, Pid 28253]: Sending Ctrl Message for cn-jenkins-deploy-platform-ansible-os-4080-3:contrail-dns:0:Control count 1 2025-09-11 Thu 04:18:37:731.529 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484345321216, Pid 28253]: Processing scm::EvSandeshSend in state ClientInit 2025-09-11 Thu 04:18:37:731.540 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484345321216, Pid 28253]: ClientInit : EvSandeshSend : NodeStatusUVE 2025-09-11 Thu 04:18:37:731.564 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484345321216, Pid 28253]: SANDESH: Send FAILED: 1757564317731375 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-ansible-os-4080-3 process_status= [ [ [ module_id = contrail-dns instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.48:8086, ] ] status = Initializing description = Connect : EvTcpConnected ], [ type = Database name = Cassandra server_addrs= [ [ (*_iter6) = 10.0.0.254:9041, (*_iter6) = 10.0.0.22:9041, (*_iter6) = 10.0.0.48:9041, ] ] status = Up description = Established Cassandra connection ], [ type = Database name = RabbitMQ server_addrs= [ [ (*_iter6) = 10.0.0.254:5673, (*_iter6) = 10.0.0.22:5673, (*_iter6) = 10.0.0.48:5673, ] ] status = Up description = RabbitMQ connection established ], ] ] flag_infos= [ [ ] ] description = Collector connection down ], ] ] ] 2025-09-11 Thu 04:18:37:731.572 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484345321216, Pid 28253]: Received UVE message in wrong state : NodeStatusUVE 2025-09-11 Thu 04:18:37:731.580 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484345321216, Pid 28253]: Processing scm::EvSandeshSend in state ClientInit 2025-09-11 Thu 04:18:37:731.586 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484345321216, Pid 28253]: ClientInit : EvSandeshSend : SandeshCtrlClientToServer 2025-09-11 Thu 04:18:37:731.596 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484345321216, Pid 28253]: Processing scm::EvSandeshSend in state ClientInit 2025-09-11 Thu 04:18:37:731.602 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484345321216, Pid 28253]: ClientInit : EvSandeshSend : SandeshModuleClientTrace 2025-09-11 Thu 04:18:37:731.763 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484345321216, Pid 28253]: SANDESH: Send FAILED: 1757564317731517 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-ansible-os-4080-3:Control:contrail-dns:0 client_info= [ status = ClientInit successful_connections = 1 pid = 28253 http_port = 8092 start_time = 1757564225184899 collector_name = collector_ip = 10.0.0.48:8086 collector_list= [ [ (*_iter40) = 10.0.0.254:8086, (*_iter40) = 10.0.0.48:8086, (*_iter40) = 10.0.0.22:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ConfigClientMgrDebug [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 197 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigurationMalformedReferenceWarningLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 10 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = DnsOperational [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 14 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapLinkOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 80 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapNodeOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 90 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 21 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 30 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 8 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveDnsInfo [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 401, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 52, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-09-11 Thu 04:18:37:731.785 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484345321216, Pid 28253]: Received UVE message in wrong state : SandeshModuleClientTrace 2025-09-11 Thu 04:18:37:733.679 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484429702912, Pid 28253]: OnMessage control in state: ClientInit 2025-09-11 Thu 04:18:37:733.736 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484429702912, Pid 28253]: Processing scm::EvSandeshMessageRecv in state ClientInit 2025-09-11 Thu 04:18:37:733.747 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484429702912, Pid 28253]: ClientInit : EvSandeshMessageRecv 2025-09-11 Thu 04:18:37:733.782 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484429702912, Pid 28253]: Received Ctrl Message with size 0 2025-09-11 Thu 04:18:37:733.802 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484429702912, Pid 28253]: SyncUVE Syncing ObjectDns val name = cn-jenkins-deploy-platform-ansible-os-4080-3 start_time = 1757564225378784 build_info = {"build-info":[{"build-version":"v0+nightly.r24.1.centos","build-time":"2025-09-11 00:09:15.426696","build-user":"root","build-hostname":"jenkins-build-prepare-sandbox-10925-1.novalocal","build-id":"v0+nightly.r24.1.centos-nightly_r24.1_centos.el7@contrail","build-number":"unknown"}]} collector = 10.0.0.254:8086 self_ip_list= [ [ (*_iter12) = 10.20.0.14, ] ] proxy seq 1 2025-09-11 Thu 04:18:37:733.815 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484429702912, Pid 28253]: SyncAllMaps for DnsState:0 period 0 cycle 0 without seqno , total = 1 2025-09-11 Thu 04:18:37:734.032 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484429702912, Pid 28253]: SyncUVE Syncing ObjectGeneratorInfo val name = cn-jenkins-deploy-platform-ansible-os-4080-3:Control:contrail-dns:0 client_info= [ status = ClientInit successful_connections = 1 pid = 28253 http_port = 8092 start_time = 1757564225184899 collector_name = collector_ip = 10.0.0.48:8086 collector_list= [ [ (*_iter40) = 10.0.0.254:8086, (*_iter40) = 10.0.0.48:8086, (*_iter40) = 10.0.0.22:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] sm_queue_count = 0 sm_stats= [ ev_stats= [ [ [ event = scm::EvCollectorUpdate enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvIdleHoldTimerExpired enqueues = 6 dequeues = 6 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvSandeshSend enqueues = 48 dequeues = 48 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvStart enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpConnectFail enqueues = 7 dequeues = 7 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpDeleteSession enqueues = 7 dequeues = 7 enqueue_fails = 0 dequeue_fails = 0 ], [ event = enqueues = 70 dequeues = 70 enqueue_fails = 0 dequeue_fails = 0 ], ] ] state = Idle state_since = 1757564287727958 last_state = Idle last_event = scm::EvSandeshSend last_event_at = 1757564287730338 ] msg_type_agg= [ [ _iter58->first = ConfigClientMgrDebug [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 197 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigurationMalformedReferenceWarningLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 10 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = DnsOperational [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 14 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapLinkOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 80 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapNodeOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 90 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 21 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 30 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 8 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveDnsInfo [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 401, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 52, _iter59->first = sent _iter59->second = 0, ] ] max_sm_queue_count = 5 proxy seq 32 2025-09-11 Thu 04:18:37:734.058 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484429702912, Pid 28253]: SyncAllMaps for ModuleClientState:0 period 0 cycle 0 without seqno , total = 1 2025-09-11 Thu 04:18:37:734.085 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484429702912, Pid 28253]: SyncUVE Syncing ObjectDns val name = cn-jenkins-deploy-platform-ansible-os-4080-3 process_status= [ [ [ module_id = contrail-dns instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.48:8086, ] ] status = Initializing description = Connect : EvTcpConnected ], [ type = Database name = Cassandra server_addrs= [ [ (*_iter6) = 10.0.0.254:9041, (*_iter6) = 10.0.0.22:9041, (*_iter6) = 10.0.0.48:9041, ] ] status = Up description = Established Cassandra connection ], [ type = Database name = RabbitMQ server_addrs= [ [ (*_iter6) = 10.0.0.254:5673, (*_iter6) = 10.0.0.22:5673, (*_iter6) = 10.0.0.48:5673, ] ] status = Up description = RabbitMQ connection established ], ] ] flag_infos= [ [ ] ] description = Collector connection down ], ] ] proxy seq 22 2025-09-11 Thu 04:18:37:734.097 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484429702912, Pid 28253]: SyncAllMaps for NodeStatus:0 period 0 cycle 0 without seqno , total = 1 2025-09-11 Thu 04:18:37:734.107 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484429702912, Pid 28253]: SyncAllMaps for PeerFlapData:0 period 0 cycle 0 without seqno , total = 0 2025-09-11 Thu 04:18:37:734.114 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484429702912, Pid 28253]: SyncAllMaps for PeerStatsData:0 period 0 cycle 0 without seqno , total = 0 2025-09-11 Thu 04:18:37:734.122 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484429702912, Pid 28253]: SyncAllMaps for XmppPeerInfoData:0 period 0 cycle 0 without seqno , total = 0 2025-09-11 Thu 04:18:37:734.155 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484429702912, Pid 28253]: Established 2025-09-11 Thu 04:20:12:945.236 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484315932416, Pid 28253]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 571 2025-09-11 Thu 04:20:13:000.616 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484311734016, Pid 28253]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 571 2025-09-11 Thu 04:20:13:100.647 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484320130816, Pid 28253]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 571 2025-09-11 Thu 04:20:13:100.822 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484349519616, Pid 28253]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating virtual-router:default-global-system-config:an-jenkins-deploy-platform-ansible-os-4080-2 controller/src/ifmap/ifmap_server_table.cc 95 2025-09-11 Thu 04:20:13:100.877 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484349519616, Pid 28253]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-virtual-router controller/src/ifmap/ifmap_server_table.cc 130 2025-09-11 Thu 04:20:13:158.727 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484425504512, Pid 28253]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 571 2025-09-11 Thu 04:20:13:158.869 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484425504512, Pid 28253]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating virtual-router:default-global-system-config:an-jenkins-deploy-platform-ansible-os-4080-1 controller/src/ifmap/ifmap_server_table.cc 95 2025-09-11 Thu 04:20:13:158.904 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484425504512, Pid 28253]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-virtual-router controller/src/ifmap/ifmap_server_table.cc 130 2025-09-11 Thu 04:20:13:473.713 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484315932416, Pid 28253]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 571 2025-09-11 Thu 04:20:13:473.893 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484315932416, Pid 28253]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating virtual-machine-interface:default-global-system-config:an-jenkins-deploy-platform-ansible-os-4080-2:vhost0 controller/src/ifmap/ifmap_server_table.cc 95 2025-09-11 Thu 04:20:13:473.942 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484315932416, Pid 28253]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating virtual-router-virtual-machine-interface controller/src/ifmap/ifmap_server_table.cc 130 2025-09-11 Thu 04:20:13:473.976 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484315932416, Pid 28253]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating virtual-machine-interface-virtual-network controller/src/ifmap/ifmap_server_table.cc 130 2025-09-11 Thu 04:20:13:541.453 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484320130816, Pid 28253]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 571 2025-09-11 Thu 04:20:13:592.003 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484429702912, Pid 28253]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 571 2025-09-11 Thu 04:20:13:592.150 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484429702912, Pid 28253]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating virtual-machine-interface:default-global-system-config:an-jenkins-deploy-platform-ansible-os-4080-1:vhost0 controller/src/ifmap/ifmap_server_table.cc 95 2025-09-11 Thu 04:20:13:592.185 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484429702912, Pid 28253]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating virtual-router-virtual-machine-interface controller/src/ifmap/ifmap_server_table.cc 130 2025-09-11 Thu 04:20:13:592.207 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484429702912, Pid 28253]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating virtual-machine-interface-virtual-network controller/src/ifmap/ifmap_server_table.cc 130 2025-09-11 Thu 04:20:13:659.222 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484332726016, Pid 28253]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 571 2025-09-11 Thu 04:20:14:009.229 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484429702912, Pid 28253]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 571 2025-09-11 Thu 04:21:35:530.943 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484585629824, Pid 28253]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.20.0.14:8092::10.20.0.14:33474 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 254 2025-09-11 Thu 04:21:36:045.097 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484585629824, Pid 28253]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.20.0.14:8092::10.20.0.14:33488 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 254 2025-09-11 Thu 04:21:36:159.511 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484585629824, Pid 28253]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.20.0.14:8092::10.20.0.254:38684 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 254 2025-09-11 Thu 04:21:36:236.504 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484585629824, Pid 28253]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.20.0.14:8092::10.20.0.17:46112 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 254 2025-09-11 Thu 04:21:36:348.764 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484585629824, Pid 28253]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.20.0.14:8092::10.20.0.254:38688 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 254 2025-09-11 Thu 04:21:36:864.013 UTC cn-jenkins-deploy-platform-ansible-os-4080-3 [Thread 140484585629824, Pid 28253]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.20.0.14:8092::10.20.0.17:44210 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 254