2025-06-02 Mon 02:06:40:958.901 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241507256448, Pid 1126454]: DnsAgent [SYS_INFO]: VersionInfoLog: {"build-info":[{"build-version":"nightly","build-time":"2025-06-02 00:23:40.800365","build-user":"root","build-hostname":"jenkins-build-prepare-sandbox-9311-1.novalocal","build-id":"unknown","build-number":"unknown"}]} src/contrail-common/base/misc_utils.cc 41 2025-06-02 Mon 02:06:40:959.863 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241507256448, Pid 1126454]: SANDESH: ROLE : Generator 2025-06-02 Mon 02:06:40:959.895 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241507256448, Pid 1126454]: SANDESH: MODULE : contrail-dns 2025-06-02 Mon 02:06:40:959.903 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241507256448, Pid 1126454]: SANDESH: SOURCE : cn-jenkins-deploy-platform-juju-os-542-1 2025-06-02 Mon 02:06:40:959.910 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241507256448, Pid 1126454]: SANDESH: NODE TYPE : Control 2025-06-02 Mon 02:06:40:959.917 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241507256448, Pid 1126454]: SANDESH: INSTANCE ID : 0 2025-06-02 Mon 02:06:40:959.927 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241507256448, Pid 1126454]: SANDESH: HTTP SERVER PORT : 8092 2025-06-02 Mon 02:06:40:959.935 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241507256448, Pid 1126454]: SANDESH: System Log Send Rate Limit: 0 -> 0 2025-06-02 Mon 02:06:40:961.856 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241507256448, Pid 1126454]: HTTP Introspect Init 2025-06-02 Mon 02:06:40:964.584 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241507256448, Pid 1126454]: TCP [SYS_DEBUG]: TcpServerMessageLog: Server 0.0.0.0:8092 Initialization complete src/contrail-common/io/tcp_server.cc 137 2025-06-02 Mon 02:06:40:964.754 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241507256448, Pid 1126454]: Sandesh Http Server Port: 8092 2025-06-02 Mon 02:06:40:966.099 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241507256448, Pid 1126454]: SANDESH: NOT Writing http_port 8092TO : /tmp/contrail-dns.1126320.http_port 2025-06-02 Mon 02:06:40:966.118 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241507256448, Pid 1126454]: SANDESH: CONNECT TO COLLECTOR: 1 2025-06-02 Mon 02:06:40:967.050 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241507256448, Pid 1126454]: Idle 2025-06-02 Mon 02:06:40:968.382 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241507256448, Pid 1126454]: SANDESH: No Client: 1748830000967249 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-juju-os-542-1:Control:contrail-dns:0 client_info= [ status = Idle successful_connections = 0 pid = 1126454 http_port = 8092 start_time = 1748830000967073 collector_name = collector_ip = 0.0.0.0:0 collector_list= [ [ ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 0, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 1, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 0, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-06-02 Mon 02:06:40:969.201 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241507256448, Pid 1126454]: SANDESH: Logging: DISABLED -> ENABLED 2025-06-02 Mon 02:06:40:969.236 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241507256448, Pid 1126454]: SANDESH: Logging: LEVEL: [ INVALID ] -> [ SYS_DEBUG ] log4level: [ NOTSET ] -> [ DEBUG ] 2025-06-02 Mon 02:06:40:969.309 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241349191424, Pid 1126454]: Processing scm::EvStart in state Idle 2025-06-02 Mon 02:06:40:969.371 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241349191424, Pid 1126454]: Disconnect 2025-06-02 Mon 02:06:40:969.477 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241349191424, Pid 1126454]: Processing scm::EvCollectorUpdate in state Disconnect 2025-06-02 Mon 02:06:40:969.810 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241349191424, Pid 1126454]: Connect : Start Connect timer 10.0.0.129:8086 2025-06-02 Mon 02:06:40:969.939 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241349191424, Pid 1126454]: Processing scm::EvSandeshSend in state Connect 2025-06-02 Mon 02:06:40:975.546 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241349191424, Pid 1126454]: SANDESH: Send FAILED: 1748830000969460 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-juju-os-542-1:Control:contrail-dns:0 client_info= [ status = Disconnect successful_connections = 0 pid = 1126454 http_port = 8092 start_time = 1748830000967073 collector_name = collector_ip = 0.0.0.0:0 collector_list= [ [ ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 1, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 0, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-06-02 Mon 02:06:40:975.611 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241349191424, Pid 1126454]: Wrong state: Connect for event: EvSandeshSend message: SandeshModuleClientTrace 2025-06-02 Mon 02:06:40:975.736 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241349191424, Pid 1126454]: Processing scm::EvSandeshSend in state Connect 2025-06-02 Mon 02:06:40:975.872 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241349191424, Pid 1126454]: SANDESH: Send FAILED: 1748830000969541 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-juju-os-542-1:Control:contrail-dns:0 client_info= [ status = Disconnect successful_connections = 0 pid = 1126454 http_port = 8092 start_time = 1748830000967073 collector_name = collector_ip = 10.0.0.129:8086 collector_list= [ [ (*_iter40) = 10.0.0.129:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 1, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 0, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-06-02 Mon 02:06:40:975.886 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241349191424, Pid 1126454]: Wrong state: Connect for event: EvSandeshSend message: SandeshModuleClientTrace 2025-06-02 Mon 02:06:40:975.896 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241349191424, Pid 1126454]: Processing scm::EvSandeshSend in state Connect 2025-06-02 Mon 02:06:40:975.997 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241349191424, Pid 1126454]: SANDESH: Send FAILED: 1748830000969923 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-juju-os-542-1:Control:contrail-dns:0 client_info= [ status = Connect successful_connections = 0 pid = 1126454 http_port = 8092 start_time = 1748830000967073 collector_name = collector_ip = 10.0.0.129:8086 collector_list= [ [ (*_iter40) = 10.0.0.129:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 1, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 0, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-06-02 Mon 02:06:40:976.007 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241349191424, Pid 1126454]: Wrong state: Connect for event: EvSandeshSend message: SandeshModuleClientTrace 2025-06-02 Mon 02:06:41:402.644 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241507256448, Pid 1126454]: TCP [SYS_DEBUG]: TcpServerMessageLog: Server 10.0.0.129:8093 Initialization complete src/contrail-common/io/tcp_server.cc 137 2025-06-02 Mon 02:06:41:402.794 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241507256448, Pid 1126454]: DnsAgent [SYS_NOTICE]: DnsOperational: Bulk sync has been started controller/src/dns/main.cc 403 2025-06-02 Mon 02:06:41:402.928 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140240547215104, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Config Client Mgr SM: Start RabbitMqReader and init Database src/contrail-common/config-client-mgr/config_client_manager.cc 303 2025-06-02 Mon 02:06:41:404.052 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140240547215104, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: RabbitMQ SM: RabbitMQ connection down src/contrail-common/config-client-mgr/config_amqp_client.cc 158 2025-06-02 Mon 02:06:41:404.101 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140240547215104, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: RabbitMQ SM: Connect to Rabbit MQ with queue_delete TRUE src/contrail-common/config-client-mgr/config_amqp_client.cc 167 2025-06-02 Mon 02:06:41:404.105 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241298810624, Pid 1126454]: Processing scm::EvSandeshSend in state Connect 2025-06-02 Mon 02:06:41:404.184 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241298810624, Pid 1126454]: SANDESH: Send FAILED: 1748830001403993 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-juju-os-542-1 process_status= [ [ [ module_id = contrail-dns instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.129:8086, ] ] status = Initializing description = Disconnect : EvCollectorUpdate -> Connect ], [ type = Database name = RabbitMQ server_addrs= [ [ (*_iter6) = 10.0.0.129:5673, ] ] status = Down description = RabbitMQ connection down ], ] ] flag_infos= [ [ ] ] description = Number of connections:2, Expected:3 Missing: Database:Cassandra ], ] ] ] 2025-06-02 Mon 02:06:41:404.238 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241298810624, Pid 1126454]: Wrong state: Connect for event: EvSandeshSend message: NodeStatusUVE 2025-06-02 Mon 02:06:41:415.191 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140240547215104, Pid 1126454]: ConfigClientManager [SYS_WARN]: ConfigClientMgrWarning: RabbitMQ SM: Caught exception while connecting to RabbitMQ: 10.0.0.129:5673 : Error in opening SSL/TLS connection for socket src/contrail-common/config-client-mgr/config_amqp_client.cc 220 2025-06-02 Mon 02:06:46:415.786 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140240547215104, Pid 1126454]: ConfigClientManager [SYS_WARN]: ConfigClientMgrWarning: RabbitMQ SM: Caught exception while connecting to RabbitMQ: 10.0.0.129:5673 : Error in opening SSL/TLS connection for socket src/contrail-common/config-client-mgr/config_amqp_client.cc 220 2025-06-02 Mon 02:06:51:416.681 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140240547215104, Pid 1126454]: ConfigClientManager [SYS_WARN]: ConfigClientMgrWarning: RabbitMQ SM: Caught exception while connecting to RabbitMQ: 10.0.0.129:5673 : Error in opening SSL/TLS connection for socket src/contrail-common/config-client-mgr/config_amqp_client.cc 220 2025-06-02 Mon 02:06:56:417.478 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140240547215104, Pid 1126454]: ConfigClientManager [SYS_WARN]: ConfigClientMgrWarning: RabbitMQ SM: Caught exception while connecting to RabbitMQ: 10.0.0.129:5673 : Error in opening SSL/TLS connection for socket src/contrail-common/config-client-mgr/config_amqp_client.cc 220 2025-06-02 Mon 02:07:01:418.716 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140240547215104, Pid 1126454]: ConfigClientManager [SYS_WARN]: ConfigClientMgrWarning: RabbitMQ SM: Caught exception while connecting to RabbitMQ: 10.0.0.129:5673 : Error in opening SSL/TLS connection for socket src/contrail-common/config-client-mgr/config_amqp_client.cc 220 2025-06-02 Mon 02:07:06:457.962 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140240547215104, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: RabbitMQ SM: RabbitMQ connection established src/contrail-common/config-client-mgr/config_amqp_client.cc 151 2025-06-02 Mon 02:07:06:458.012 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140240547215104, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Config Client Mgr SM: Init Database src/contrail-common/config-client-mgr/config_client_manager.cc 307 2025-06-02 Mon 02:07:06:458.054 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140240547215104, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Lost Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 580 2025-06-02 Mon 02:07:06:458.081 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140240547215104, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Db Init src/contrail-common/config-client-mgr/config_cassandra_client.cc 87 2025-06-02 Mon 02:07:06:460.671 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241319802624, Pid 1126454]: Processing scm::EvSandeshSend in state Connect 2025-06-02 Mon 02:07:06:460.733 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241319802624, Pid 1126454]: SANDESH: Send FAILED: 1748830026457917 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-juju-os-542-1 process_status= [ [ [ module_id = contrail-dns instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.129:8086, ] ] status = Initializing description = Disconnect : EvCollectorUpdate -> Connect ], [ type = Database name = RabbitMQ server_addrs= [ [ (*_iter6) = 10.0.0.129:5673, ] ] status = Up description = RabbitMQ connection established ], ] ] flag_infos= [ [ ] ] description = Number of connections:2, Expected:3 Missing: Database:Cassandra ], ] ] ] 2025-06-02 Mon 02:07:06:460.744 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241319802624, Pid 1126454]: Wrong state: Connect for event: EvSandeshSend message: NodeStatusUVE 2025-06-02 Mon 02:07:06:460.755 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241319802624, Pid 1126454]: Processing scm::EvSandeshSend in state Connect 2025-06-02 Mon 02:07:06:460.786 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241319802624, Pid 1126454]: SANDESH: Send FAILED: 1748830026458044 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-juju-os-542-1 process_status= [ [ [ module_id = contrail-dns instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.129:8086, ] ] status = Initializing description = Disconnect : EvCollectorUpdate -> Connect ], [ type = Database name = Cassandra server_addrs= [ [ (*_iter6) = 10.0.0.129:9041, ] ] status = Down description = Lost Cassandra connection ], [ type = Database name = RabbitMQ server_addrs= [ [ (*_iter6) = 10.0.0.129:5673, ] ] status = Up description = RabbitMQ connection established ], ] ] flag_infos= [ [ ] ] description = Collector, Database:Cassandra connection down ], ] ] ] 2025-06-02 Mon 02:07:06:460.794 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241319802624, Pid 1126454]: Wrong state: Connect for event: EvSandeshSend message: NodeStatusUVE 2025-06-02 Mon 02:07:06:665.913 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140240547215104, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Db SetTableSpace src/contrail-common/config-client-mgr/config_cassandra_client.cc 95 2025-06-02 Mon 02:07:06:667.795 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140240547215104, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigCassInitErrorMessage: Setting database keyspace failed src/contrail-common/config-client-mgr/config_cassandra_client.cc 99 2025-06-02 Mon 02:07:06:667.834 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140240547215104, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: DB uninit src/contrail-common/config-client-mgr/config_cassandra_client.cc 122 2025-06-02 Mon 02:07:11:668.284 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140240547215104, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Db Init src/contrail-common/config-client-mgr/config_cassandra_client.cc 87 2025-06-02 Mon 02:07:11:748.048 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140240547215104, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Db SetTableSpace src/contrail-common/config-client-mgr/config_cassandra_client.cc 95 2025-06-02 Mon 02:07:11:749.348 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140240547215104, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigCassInitErrorMessage: Setting database keyspace failed src/contrail-common/config-client-mgr/config_cassandra_client.cc 99 2025-06-02 Mon 02:07:11:749.388 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140240547215104, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: DB uninit src/contrail-common/config-client-mgr/config_cassandra_client.cc 122 2025-06-02 Mon 02:07:16:751.136 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140240547215104, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Db Init src/contrail-common/config-client-mgr/config_cassandra_client.cc 87 2025-06-02 Mon 02:07:17:491.294 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140240547215104, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Db SetTableSpace src/contrail-common/config-client-mgr/config_cassandra_client.cc 95 2025-06-02 Mon 02:07:17:549.158 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140240547215104, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigCassInitErrorMessage: Setting database keyspace failed src/contrail-common/config-client-mgr/config_cassandra_client.cc 99 2025-06-02 Mon 02:07:17:549.191 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140240547215104, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: DB uninit src/contrail-common/config-client-mgr/config_cassandra_client.cc 122 2025-06-02 Mon 02:07:22:567.625 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140240547215104, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Db Init src/contrail-common/config-client-mgr/config_cassandra_client.cc 87 2025-06-02 Mon 02:07:23:904.810 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140240547215104, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Db SetTableSpace src/contrail-common/config-client-mgr/config_cassandra_client.cc 95 2025-06-02 Mon 02:07:23:907.126 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140240547215104, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigCassInitErrorMessage: Setting database keyspace failed src/contrail-common/config-client-mgr/config_cassandra_client.cc 99 2025-06-02 Mon 02:07:23:907.172 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140240547215104, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: DB uninit src/contrail-common/config-client-mgr/config_cassandra_client.cc 122 2025-06-02 Mon 02:07:28:910.713 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140240547215104, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Db Init src/contrail-common/config-client-mgr/config_cassandra_client.cc 87 2025-06-02 Mon 02:07:29:418.096 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140240547215104, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Db SetTableSpace src/contrail-common/config-client-mgr/config_cassandra_client.cc 95 2025-06-02 Mon 02:07:29:450.394 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140240547215104, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Db UseColumnFamily uuidTableName src/contrail-common/config-client-mgr/config_cassandra_client.cc 104 2025-06-02 Mon 02:07:29:450.475 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140240547215104, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: DB uninit src/contrail-common/config-client-mgr/config_cassandra_client.cc 122 2025-06-02 Mon 02:07:34:466.986 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140240547215104, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Db Init src/contrail-common/config-client-mgr/config_cassandra_client.cc 87 2025-06-02 Mon 02:07:34:879.402 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140240547215104, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Db SetTableSpace src/contrail-common/config-client-mgr/config_cassandra_client.cc 95 2025-06-02 Mon 02:07:34:881.525 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140240547215104, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Db UseColumnFamily uuidTableName src/contrail-common/config-client-mgr/config_cassandra_client.cc 104 2025-06-02 Mon 02:07:34:881.571 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140240547215104, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Db UseColumnFamily fqnTableName src/contrail-common/config-client-mgr/config_cassandra_client.cc 110 2025-06-02 Mon 02:07:34:881.666 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140240547215104, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:07:34:881.682 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140240547215104, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: BulkDataSync Started src/contrail-common/config-client-mgr/config_cassandra_client.cc 393 2025-06-02 Mon 02:07:34:882.280 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140240543016704, Pid 1126454]: Processing scm::EvSandeshSend in state Connect 2025-06-02 Mon 02:07:34:882.346 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140240543016704, Pid 1126454]: SANDESH: Send FAILED: 1748830054881617 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-juju-os-542-1 process_status= [ [ [ module_id = contrail-dns instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.129:8086, ] ] status = Initializing description = Disconnect : EvCollectorUpdate -> Connect ], [ type = Database name = Cassandra server_addrs= [ [ (*_iter6) = 10.0.0.129:9041, ] ] status = Up description = Established Cassandra connection ], [ type = Database name = RabbitMQ server_addrs= [ [ (*_iter6) = 10.0.0.129:5673, ] ] status = Up description = RabbitMQ connection established ], ] ] flag_infos= [ [ ] ] description = Collector connection down ], ] ] ] 2025-06-02 Mon 02:07:34:882.375 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140240543016704, Pid 1126454]: Wrong state: Connect for event: EvSandeshSend message: NodeStatusUVE 2025-06-02 Mon 02:07:34:884.171 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241319802624, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:07:34:886.386 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241319802624, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:07:34:890.125 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241319802624, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:07:34:891.633 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241319802624, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:07:34:893.096 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241319802624, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:07:34:895.249 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241319802624, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:07:34:897.911 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241319802624, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:07:34:900.673 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241319802624, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:07:34:903.932 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241319802624, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:07:34:908.315 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241319802624, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:07:34:910.436 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241319802624, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:07:34:911.516 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241319802624, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:07:34:912.748 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241319802624, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:07:34:916.667 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241319802624, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:07:34:932.943 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241319802624, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:07:34:950.971 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241319802624, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:07:34:953.504 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241319802624, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:07:34:955.524 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241319802624, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:07:34:957.693 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241319802624, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:07:34:962.610 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241319802624, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:07:34:964.664 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241319802624, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:07:34:966.741 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241319802624, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:07:34:976.409 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241319802624, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:07:34:981.851 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241319802624, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:07:34:984.991 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241319802624, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:07:34:987.930 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241319802624, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:07:34:991.708 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241319802624, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:07:34:994.971 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241319802624, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:07:34:998.482 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241319802624, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:07:35:008.390 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241319802624, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:07:35:016.204 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241319802624, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:07:35:018.841 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241319802624, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:07:35:025.357 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241319802624, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:07:35:031.994 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241319802624, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:07:35:033.944 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241319802624, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:07:35:043.955 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241319802624, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:07:35:051.688 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241319802624, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:07:35:060.317 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241319802624, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:07:35:062.524 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241319802624, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:07:35:082.558 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241319802624, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:07:35:097.437 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241319802624, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:07:35:105.305 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241319802624, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:07:35:112.266 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241319802624, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:07:35:117.161 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241319802624, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:07:35:120.244 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241319802624, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:07:35:125.504 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241319802624, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:07:35:143.049 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241319802624, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:07:35:173.817 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241319802624, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:07:35:175.454 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241319802624, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:07:35:239.384 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241319802624, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:07:35:245.029 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241319802624, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:07:35:246.783 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241319802624, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:07:35:248.209 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241319802624, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:07:35:251.249 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241319802624, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:07:35:253.989 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241319802624, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:07:35:255.107 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241319802624, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:07:35:256.226 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241319802624, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:07:35:257.585 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241319802624, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:07:35:258.698 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241319802624, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:07:35:259.834 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241319802624, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:07:35:260.828 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241319802624, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:07:35:261.935 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241319802624, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:07:35:263.936 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241319802624, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:07:35:268.711 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241319802624, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:07:35:283.017 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241319802624, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:07:35:297.753 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241319802624, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:07:35:305.249 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241319802624, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:07:35:337.633 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241319802624, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:07:35:339.119 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241319802624, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:07:35:341.742 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241319802624, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:07:35:343.960 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241319802624, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:07:35:346.075 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241319802624, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:07:35:348.055 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241319802624, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:07:35:351.219 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241319802624, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:07:35:353.040 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241319802624, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:07:35:355.093 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241319802624, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:07:35:356.862 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241319802624, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:07:35:358.359 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241319802624, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:07:35:359.325 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241319802624, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:07:35:360.414 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241319802624, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:07:35:362.039 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241319802624, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:07:35:363.334 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241319802624, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:07:35:365.948 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241319802624, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:07:35:374.391 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241319802624, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:07:35:375.610 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241319802624, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:07:35:376.873 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241319802624, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:07:35:378.166 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241319802624, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:07:35:379.434 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241319802624, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:07:35:381.123 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241319802624, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:07:35:383.027 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241319802624, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:07:35:384.011 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241319802624, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:07:35:385.096 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241319802624, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:07:35:386.289 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241319802624, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:07:35:387.451 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241319802624, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:07:35:397.271 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241319802624, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:07:35:399.602 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241319802624, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:07:35:400.888 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241319802624, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:07:35:402.660 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241319802624, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:07:35:403.690 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241319802624, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:07:35:405.198 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241319802624, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:07:35:406.084 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241319802624, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:07:35:408.362 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241319802624, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:07:35:409.313 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241319802624, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:07:35:411.602 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241319802624, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:07:35:412.645 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241319802624, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:07:35:413.877 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241319802624, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:07:35:414.746 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241319802624, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:07:35:416.021 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241319802624, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:07:35:417.040 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241319802624, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:07:35:432.079 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241319802624, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:07:35:433.420 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241319802624, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:07:35:435.521 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241319802624, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:07:35:436.586 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241319802624, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:07:35:459.293 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241319802624, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:07:35:460.878 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241319802624, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:07:35:463.360 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241319802624, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:07:35:464.456 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241319802624, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:07:35:466.181 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241319802624, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:07:35:466.282 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241319802624, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: One reader finished BulkSync src/contrail-common/config-client-mgr/config_cassandra_client.cc 558 2025-06-02 Mon 02:07:35:466.304 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241319802624, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: One reader finished BulkSync src/contrail-common/config-client-mgr/config_cassandra_client.cc 558 2025-06-02 Mon 02:07:35:466.313 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241319802624, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: One reader finished BulkSync src/contrail-common/config-client-mgr/config_cassandra_client.cc 558 2025-06-02 Mon 02:07:35:466.321 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241319802624, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: One reader finished BulkSync src/contrail-common/config-client-mgr/config_cassandra_client.cc 558 2025-06-02 Mon 02:07:35:466.330 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241319802624, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: One reader finished BulkSync src/contrail-common/config-client-mgr/config_cassandra_client.cc 558 2025-06-02 Mon 02:07:35:466.334 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241344993024, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: One reader finished BulkSync src/contrail-common/config-client-mgr/config_cassandra_client.cc 558 2025-06-02 Mon 02:07:35:466.389 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241344993024, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: BulkSyncDone by all readers src/contrail-common/config-client-mgr/config_cassandra_client.cc 554 2025-06-02 Mon 02:07:35:474.282 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241344993024, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Config Client Mgr SM: End of RIB computed and notification sent src/contrail-common/config-client-mgr/config_client_manager.cc 197 2025-06-02 Mon 02:07:35:474.507 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241303009024, Pid 1126454]: Processing scm::EvSandeshSend in state Connect 2025-06-02 Mon 02:07:35:474.576 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241303009024, Pid 1126454]: SANDESH: Send FAILED: 1748830055474352 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-juju-os-542-1 process_status= [ [ [ module_id = contrail-dns instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.129:8086, ] ] status = Initializing description = Disconnect : EvCollectorUpdate -> Connect ], [ type = Database name = Cassandra server_addrs= [ [ (*_iter6) = 10.0.0.129:9041, ] ] status = Up description = Established Cassandra connection ], [ type = Database name = RabbitMQ server_addrs= [ [ (*_iter6) = 10.0.0.129:5673, ] ] status = Up description = RabbitMQ connection established ], ] ] flag_infos= [ [ ] ] description = Collector connection down ], ] ] ] 2025-06-02 Mon 02:07:35:474.587 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241303009024, Pid 1126454]: Wrong state: Connect for event: EvSandeshSend message: NodeStatusUVE 2025-06-02 Mon 02:07:35:466.338 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241319802624, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: One reader finished BulkSync src/contrail-common/config-client-mgr/config_cassandra_client.cc 558 2025-06-02 Mon 02:07:35:477.829 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140240547215104, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: TRUE src/contrail-common/config-client-mgr/config_client_manager.cc 225 2025-06-02 Mon 02:07:35:481.920 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241507256448, Pid 1126454]: DnsAgent [SYS_NOTICE]: DnsOperational: EndOfRib point was reached, IF-MAP objects have been created controller/src/dns/main.cc 86 2025-06-02 Mon 02:07:35:482.158 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241507256448, Pid 1126454]: DnsAgent [SYS_NOTICE]: DnsOperational: The IF-MAP database table has been updated: __ifmap__.virtual_DNS.0, counting 0 elements controller/src/dns/main.cc 98 2025-06-02 Mon 02:07:35:482.190 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241507256448, Pid 1126454]: DnsAgent [SYS_NOTICE]: DnsOperational: The IF-MAP database table has been updated: __ifmap__.virtual_DNS_record.0, counting 0 elements controller/src/dns/main.cc 98 2025-06-02 Mon 02:07:35:482.200 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241507256448, Pid 1126454]: DnsAgent [SYS_NOTICE]: DnsOperational: The IF-MAP database table has been updated: __ifmap__.network_ipam.0, counting 0 elements controller/src/dns/main.cc 98 2025-06-02 Mon 02:07:35:482.210 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241507256448, Pid 1126454]: DnsAgent [SYS_NOTICE]: DnsOperational: The IF-MAP database table has been updated: __ifmap__.virtual_network_network_ipam.0, counting 0 elements controller/src/dns/main.cc 98 2025-06-02 Mon 02:07:35:482.219 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241507256448, Pid 1126454]: DnsAgent [SYS_NOTICE]: DnsOperational: The IF-MAP database table has been updated: __ifmap__.global_qos_config.0, counting 0 elements controller/src/dns/main.cc 98 2025-06-02 Mon 02:07:35:482.227 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241507256448, Pid 1126454]: DnsAgent [SYS_NOTICE]: DnsOperational: VnniConfig has been updated after the bulk sync, the number of elements: 0 controller/src/dns/main.cc 109 2025-06-02 Mon 02:07:35:482.238 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241507256448, Pid 1126454]: DnsAgent [SYS_NOTICE]: DnsOperational: IpamConfig has been updated after the bulk sync, the number of elements: 0 controller/src/dns/main.cc 120 2025-06-02 Mon 02:07:35:482.250 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241507256448, Pid 1126454]: DnsAgent [SYS_NOTICE]: DnsOperational: VirtualDnsConfig has been updated after the bulk sync, the number of elements: 0 controller/src/dns/main.cc 131 2025-06-02 Mon 02:07:35:482.262 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241507256448, Pid 1126454]: DnsAgent [SYS_NOTICE]: DnsOperational: VirtualDnsRecordConfig has been updated after the bulk sync, the number of elements: 0 controller/src/dns/main.cc 142 2025-06-02 Mon 02:07:35:482.274 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241507256448, Pid 1126454]: DnsAgent [SYS_NOTICE]: DnsOperational: GlobalQosConfig has been updated after the bulk sync controller/src/dns/main.cc 152 2025-06-02 Mon 02:07:35:482.695 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241332397824, Pid 1126454]: Processing scm::EvSandeshSend in state Connect 2025-06-02 Mon 02:07:35:482.743 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241332397824, Pid 1126454]: SANDESH: Send FAILED: 1748830055482314 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-juju-os-542-1 process_status= [ [ [ module_id = contrail-dns instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.129:8086, ] ] status = Initializing description = Disconnect : EvCollectorUpdate -> Connect ], [ type = Database name = Cassandra server_addrs= [ [ (*_iter6) = 10.0.0.129:9041, ] ] status = Up description = Established Cassandra connection ], [ type = Database name = RabbitMQ server_addrs= [ [ (*_iter6) = 10.0.0.129:5673, ] ] status = Down description = RabbitMQ connection down ], ] ] flag_infos= [ [ ] ] description = Collector, Database:RabbitMQ connection down ], ] ] ] 2025-06-02 Mon 02:07:35:482.757 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241332397824, Pid 1126454]: Wrong state: Connect for event: EvSandeshSend message: NodeStatusUVE 2025-06-02 Mon 02:07:35:482.782 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241507256448, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: RabbitMQ SM: RabbitMQ connection down src/contrail-common/config-client-mgr/config_amqp_client.cc 158 2025-06-02 Mon 02:07:35:482.805 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241507256448, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: RabbitMQ SM: Connect to Rabbit MQ with queue_delete TRUE src/contrail-common/config-client-mgr/config_amqp_client.cc 167 2025-06-02 Mon 02:07:35:521.494 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241507256448, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: RabbitMQ SM: RabbitMQ connection established src/contrail-common/config-client-mgr/config_amqp_client.cc 151 2025-06-02 Mon 02:07:35:521.496 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140240547215104, Pid 1126454]: Processing scm::EvSandeshSend in state Connect 2025-06-02 Mon 02:07:35:522.004 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140240547215104, Pid 1126454]: SANDESH: Send FAILED: 1748830055520405 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-juju-os-542-1 process_status= [ [ [ module_id = contrail-dns instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.129:8086, ] ] status = Initializing description = Disconnect : EvCollectorUpdate -> Connect ], [ type = Database name = Cassandra server_addrs= [ [ (*_iter6) = 10.0.0.129:9041, ] ] status = Up description = Established Cassandra connection ], [ type = Database name = RabbitMQ server_addrs= [ [ (*_iter6) = 10.0.0.129:5673, ] ] status = Up description = RabbitMQ connection established ], ] ] flag_infos= [ [ ] ] description = Collector connection down ], ] ] ] 2025-06-02 Mon 02:07:35:522.016 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140240547215104, Pid 1126454]: Wrong state: Connect for event: EvSandeshSend message: NodeStatusUVE 2025-06-02 Mon 02:07:35:522.034 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140240547215104, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: TRUE src/contrail-common/config-client-mgr/config_client_manager.cc 225 2025-06-02 Mon 02:07:35:526.972 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241328199424, Pid 1126454]: 10.0.0.129:8086 EvConnectTimerExpired in state Connect 2025-06-02 Mon 02:07:35:527.290 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241324001024, Pid 1126454]: Processing scm::EvConnectTimerExpired in state Connect 2025-06-02 Mon 02:07:35:527.310 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241324001024, Pid 1126454]: Connect : EvConnectTimerExpired 2025-06-02 Mon 02:07:35:527.379 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241324001024, Pid 1126454]: Idle 2025-06-02 Mon 02:07:35:527.444 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241324001024, Pid 1126454]: Processing scm::EvSandeshSend in state Idle 2025-06-02 Mon 02:07:35:527.515 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241324001024, Pid 1126454]: SANDESH: Send FAILED: 1748830055527195 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-juju-os-542-1:Control:contrail-dns:0 sm_queue_count = 1 sm_stats= [ ev_stats= [ [ [ event = scm::EvCollectorUpdate enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvConnectTimerExpired enqueues = 1 dequeues = 0 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvSandeshSend enqueues = 10 dequeues = 10 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvStart enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = enqueues = 13 dequeues = 12 enqueue_fails = 0 dequeue_fails = 0 ], ] ] state = Connect state_since = 1748830000969555 last_state = Connect last_event = scm::EvSandeshSend last_event_at = 1748830055521475 ] session_stats= [ num_recv_msg = 0 num_recv_msg_fail = 0 num_recv_fail = 0 num_send_msg = 0 num_send_msg_fail = 0 num_send_buffer_fail = 0 num_wait_msgq_enqueue = 0 num_wait_msgq_dequeue = 0 num_write_ready_cb_error = 0 ] session_rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] session_tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] max_sm_queue_count = 3 ] 2025-06-02 Mon 02:07:35:527.534 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241324001024, Pid 1126454]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-06-02 Mon 02:07:35:527.542 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241324001024, Pid 1126454]: Processing scm::EvSandeshSend in state Idle 2025-06-02 Mon 02:07:35:527.680 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241324001024, Pid 1126454]: SANDESH: Send FAILED: 1748830055527263 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-juju-os-542-1:Control:contrail-dns:0 client_info= [ status = Connect successful_connections = 0 pid = 1126454 http_port = 8092 start_time = 1748830000967073 collector_name = collector_ip = 10.0.0.129:8086 collector_list= [ [ (*_iter40) = 10.0.0.129:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ConfigCassInitErrorMessage [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 4 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigClientMgrDebug [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 160 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigClientMgrWarning [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 5 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = DnsOperational [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 12 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 7 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 3 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 183, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 10, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-06-02 Mon 02:07:35:527.693 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241324001024, Pid 1126454]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-06-02 Mon 02:07:35:527.702 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241324001024, Pid 1126454]: Processing scm::EvSandeshSend in state Idle 2025-06-02 Mon 02:07:35:527.723 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241324001024, Pid 1126454]: SANDESH: Send FAILED: 1748830055527335 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-juju-os-542-1 process_status= [ [ [ module_id = contrail-dns instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvConnectTimerExpired ], [ type = Database name = Cassandra server_addrs= [ [ (*_iter6) = 10.0.0.129:9041, ] ] status = Up description = Established Cassandra connection ], [ type = Database name = RabbitMQ server_addrs= [ [ (*_iter6) = 10.0.0.129:5673, ] ] status = Up description = RabbitMQ connection established ], ] ] flag_infos= [ [ ] ] description = Collector connection down ], ] ] ] 2025-06-02 Mon 02:07:35:527.730 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241324001024, Pid 1126454]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-06-02 Mon 02:07:35:527.738 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241324001024, Pid 1126454]: Processing scm::EvTcpDeleteSession in state Idle 2025-06-02 Mon 02:07:35:527.749 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241324001024, Pid 1126454]: Processing scm::EvSandeshSend in state Idle 2025-06-02 Mon 02:07:35:527.869 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241324001024, Pid 1126454]: SANDESH: Send FAILED: 1748830055527431 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-juju-os-542-1:Control:contrail-dns:0 client_info= [ status = Idle successful_connections = 0 pid = 1126454 http_port = 8092 start_time = 1748830000967073 collector_name = collector_ip = 10.0.0.129:8086 collector_list= [ [ (*_iter40) = 10.0.0.129:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ConfigCassInitErrorMessage [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 4 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigClientMgrDebug [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 160 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigClientMgrWarning [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 5 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = DnsOperational [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 12 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 7 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 3 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 183, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 10, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-06-02 Mon 02:07:35:527.880 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241324001024, Pid 1126454]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-06-02 Mon 02:07:35:533.820 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241507256448, Pid 1126454]: TCP [SYS_ERR]: TcpSessionMessageLog: Session -(-1) > SSL Handshake failed due to error: 9 category: system message: Bad file descriptor src/contrail-common/io/ssl_server.cc 134 2025-06-02 Mon 02:07:35:535.546 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241507256448, Pid 1126454]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.0.0.129:8092::10.0.0.129:33686 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 256 2025-06-02 Mon 02:07:35:535.708 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241328199424, Pid 1126454]: DnsAgent [SYS_NOTICE]: DnsOperational: BIND named up; DNS is operational controller/src/dns/mgr/dns_mgr.cc 737 2025-06-02 Mon 02:07:35:536.760 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241507256448, Pid 1126454]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.0.0.129:8092::10.0.0.129:33694 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 256 2025-06-02 Mon 02:07:35:595.082 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241507256448, Pid 1126454]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.0.0.129:8092::10.0.0.129:33686 < Read failed due to error asio.ssl 335544539 : short read src/contrail-common/io/tcp_session.cc 504 2025-06-02 Mon 02:07:35:600.930 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241507256448, Pid 1126454]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.0.0.129:8092::10.0.0.129:33694 < Read failed due to error asio.ssl 335544539 : short read src/contrail-common/io/tcp_session.cc 504 2025-06-02 Mon 02:07:36:531.450 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241340794624, Pid 1126454]: Processing scm::EvIdleHoldTimerExpired in state Idle 2025-06-02 Mon 02:07:36:532.922 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241340794624, Pid 1126454]: Connect : Start Connect timer 10.0.0.129:8086 2025-06-02 Mon 02:07:36:533.056 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241340794624, Pid 1126454]: Processing scm::EvSandeshSend in state Connect 2025-06-02 Mon 02:07:36:533.179 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241340794624, Pid 1126454]: SANDESH: Send FAILED: 1748830056531527 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-juju-os-542-1 process_status= [ [ [ module_id = contrail-dns instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.129:8086, ] ] status = Initializing description = Idle : EvIdleHoldTimerExpired -> Connect ], [ type = Database name = Cassandra server_addrs= [ [ (*_iter6) = 10.0.0.129:9041, ] ] status = Up description = Established Cassandra connection ], [ type = Database name = RabbitMQ server_addrs= [ [ (*_iter6) = 10.0.0.129:5673, ] ] status = Up description = RabbitMQ connection established ], ] ] flag_infos= [ [ ] ] description = Collector connection down ], ] ] ] 2025-06-02 Mon 02:07:36:533.195 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241340794624, Pid 1126454]: Wrong state: Connect for event: EvSandeshSend message: NodeStatusUVE 2025-06-02 Mon 02:07:36:533.205 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241340794624, Pid 1126454]: Processing scm::EvSandeshSend in state Connect 2025-06-02 Mon 02:07:36:533.740 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241340794624, Pid 1126454]: SANDESH: Send FAILED: 1748830056533039 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-juju-os-542-1:Control:contrail-dns:0 client_info= [ status = Connect successful_connections = 0 pid = 1126454 http_port = 8092 start_time = 1748830000967073 collector_name = collector_ip = 10.0.0.129:8086 collector_list= [ [ (*_iter40) = 10.0.0.129:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ConfigCassInitErrorMessage [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 4 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigClientMgrDebug [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 160 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigClientMgrWarning [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 5 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = DnsOperational [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 13 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 8 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 6 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 5 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 189, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 14, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-06-02 Mon 02:07:36:533.772 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241340794624, Pid 1126454]: Wrong state: Connect for event: EvSandeshSend message: SandeshModuleClientTrace 2025-06-02 Mon 02:07:37:106.525 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241298810624, Pid 1126454]: DnsAgent [SYS_NOTICE]: DnsOperational: Bulk sync has been done (the DB has been synced with named) controller/src/dns/mgr/dns_mgr.cc 455 2025-06-02 Mon 02:07:38:535.094 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241340794624, Pid 1126454]: 10.0.0.129:8086 EvConnectTimerExpired in state Connect 2025-06-02 Mon 02:07:38:535.171 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241340794624, Pid 1126454]: Processing scm::EvConnectTimerExpired in state Connect 2025-06-02 Mon 02:07:38:535.182 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241340794624, Pid 1126454]: Connect : EvConnectTimerExpired 2025-06-02 Mon 02:07:38:535.306 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241340794624, Pid 1126454]: Idle 2025-06-02 Mon 02:07:38:535.399 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241340794624, Pid 1126454]: Processing scm::EvSandeshSend in state Idle 2025-06-02 Mon 02:07:38:535.439 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241340794624, Pid 1126454]: SANDESH: Send FAILED: 1748830058535231 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-juju-os-542-1 process_status= [ [ [ module_id = contrail-dns instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvConnectTimerExpired ], [ type = Database name = Cassandra server_addrs= [ [ (*_iter6) = 10.0.0.129:9041, ] ] status = Up description = Established Cassandra connection ], [ type = Database name = RabbitMQ server_addrs= [ [ (*_iter6) = 10.0.0.129:5673, ] ] status = Up description = RabbitMQ connection established ], ] ] flag_infos= [ [ ] ] description = Collector connection down ], ] ] ] 2025-06-02 Mon 02:07:38:535.517 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241340794624, Pid 1126454]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-06-02 Mon 02:07:38:536.702 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241340794624, Pid 1126454]: Processing scm::EvTcpDeleteSession in state Idle 2025-06-02 Mon 02:07:38:536.731 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241340794624, Pid 1126454]: Processing scm::EvSandeshSend in state Idle 2025-06-02 Mon 02:07:38:536.922 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241340794624, Pid 1126454]: SANDESH: Send FAILED: 1748830058535389 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-juju-os-542-1:Control:contrail-dns:0 client_info= [ status = Idle successful_connections = 0 pid = 1126454 http_port = 8092 start_time = 1748830000967073 collector_name = collector_ip = 10.0.0.129:8086 collector_list= [ [ (*_iter40) = 10.0.0.129:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ConfigCassInitErrorMessage [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 4 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigClientMgrDebug [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 160 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigClientMgrWarning [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 5 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = DnsOperational [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 14 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 9 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 7 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 5 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 190, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 16, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-06-02 Mon 02:07:38:536.945 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241340794624, Pid 1126454]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-06-02 Mon 02:07:38:536.161 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241507256448, Pid 1126454]: TCP [SYS_ERR]: TcpSessionMessageLog: Session -(-1) > SSL Handshake failed due to error: 125 category: system message: Operation canceled src/contrail-common/io/ssl_server.cc 134 2025-06-02 Mon 02:07:38:711.003 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241294612224, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:07:38:712.459 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241332397824, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating global-system-config:default-global-system-config controller/src/ifmap/ifmap_server_table.cc 95 2025-06-02 Mon 02:07:38:775.036 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241332397824, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:07:38:775.201 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241332397824, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating global-vrouter-config:default-global-system-config:default-global-vrouter-config controller/src/ifmap/ifmap_server_table.cc 95 2025-06-02 Mon 02:07:38:775.236 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241332397824, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-global-vrouter-config controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:07:38:826.104 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241344993024, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:07:38:826.234 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241344993024, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating domain:default-domain controller/src/ifmap/ifmap_server_table.cc 95 2025-06-02 Mon 02:07:38:884.516 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241303009024, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:07:38:884.668 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241303009024, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating project:default-domain:default-project controller/src/ifmap/ifmap_server_table.cc 95 2025-06-02 Mon 02:07:38:884.700 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241303009024, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating domain-project controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:07:38:900.607 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241340794624, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:07:38:900.836 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241340794624, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating network-ipam:default-domain:default-project:default-network-ipam controller/src/ifmap/ifmap_server_table.cc 95 2025-06-02 Mon 02:07:38:900.871 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241340794624, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating project-network-ipam controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:07:38:954.771 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140240543016704, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:07:38:954.920 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241315604224, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating virtual-network:default-domain:default-project:default-virtual-network controller/src/ifmap/ifmap_server_table.cc 95 2025-06-02 Mon 02:07:38:954.978 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241315604224, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating project-virtual-network controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:07:38:988.187 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241290413824, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:07:38:988.317 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241290413824, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating namespace:default-domain:default-namespace controller/src/ifmap/ifmap_server_table.cc 95 2025-06-02 Mon 02:07:38:988.348 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241290413824, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating domain-namespace controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:07:39:034.919 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241319802624, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:07:39:035.208 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241332397824, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating fabric:default-global-system-config:default-fabric controller/src/ifmap/ifmap_server_table.cc 95 2025-06-02 Mon 02:07:39:035.267 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241332397824, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-fabric controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:07:39:081.842 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241319802624, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:07:39:081.961 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241319802624, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating policy-management:default-policy-management controller/src/ifmap/ifmap_server_table.cc 95 2025-06-02 Mon 02:07:39:128.957 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241332397824, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:07:39:129.743 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241324001024, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating application-policy-set:default-policy-management:default-application-policy-set controller/src/ifmap/ifmap_server_table.cc 95 2025-06-02 Mon 02:07:39:129.807 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241324001024, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating policy-management-application-policy-set controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:07:39:181.325 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241315604224, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:07:39:181.478 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241315604224, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating application-policy-set-global-vrouter-config controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:07:39:314.362 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241307207424, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:07:39:314.663 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241307207424, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating virtual-network:default-domain:default-project:ip-fabric controller/src/ifmap/ifmap_server_table.cc 95 2025-06-02 Mon 02:07:39:314.720 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241307207424, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating project-virtual-network controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:07:39:363.393 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241319802624, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:07:39:395.890 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241336596224, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:07:39:486.987 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241311405824, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:07:39:487.202 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241311405824, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating virtual-network:default-domain:default-project:__link_local__ controller/src/ifmap/ifmap_server_table.cc 95 2025-06-02 Mon 02:07:39:487.255 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241311405824, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating project-virtual-network controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:07:39:528.944 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241324001024, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:07:39:600.323 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241294612224, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:07:39:600.541 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241319802624, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating virtual-network:default-domain:default-project:dci-network controller/src/ifmap/ifmap_server_table.cc 95 2025-06-02 Mon 02:07:39:600.657 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241319802624, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating project-virtual-network controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:07:39:673.210 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241311405824, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:07:39:734.787 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241298810624, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:07:39:760.700 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241349191424, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:07:39:764.908 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241349191424, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating alarm:default-global-system-config:system-defined-bottle-request-size-limit controller/src/ifmap/ifmap_server_table.cc 95 2025-06-02 Mon 02:07:39:764.974 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241349191424, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-alarm controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:07:39:793.620 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241298810624, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:07:39:833.609 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241294612224, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:07:39:833.846 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241311405824, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating security-group:default-domain:default-project:__no_rule__ controller/src/ifmap/ifmap_server_table.cc 95 2025-06-02 Mon 02:07:39:833.917 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241311405824, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating project-security-group controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:07:39:866.892 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241336596224, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:07:39:867.313 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241298810624, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating discovery-service-assignment:default-discovery-service-assignment controller/src/ifmap/ifmap_server_table.cc 95 2025-06-02 Mon 02:07:39:889.183 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140240543016704, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:07:39:889.963 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241298810624, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating global-qos-config:default-global-system-config:default-global-qos-config controller/src/ifmap/ifmap_server_table.cc 95 2025-06-02 Mon 02:07:39:894.583 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241298810624, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-global-qos-config controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:07:39:938.004 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241303009024, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:07:39:938.187 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241303009024, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating network-ipam:default-domain:default-project:service-chain-flat-ipam controller/src/ifmap/ifmap_server_table.cc 95 2025-06-02 Mon 02:07:39:938.220 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241303009024, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating project-network-ipam controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:07:39:968.076 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241303009024, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:07:39:968.221 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241303009024, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating tag-type:label controller/src/ifmap/ifmap_server_table.cc 95 2025-06-02 Mon 02:07:39:987.619 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241332397824, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:07:39:987.792 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241332397824, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating tag-type:application controller/src/ifmap/ifmap_server_table.cc 95 2025-06-02 Mon 02:07:40:033.286 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241311405824, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:07:40:033.478 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241311405824, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating tag-type:tier controller/src/ifmap/ifmap_server_table.cc 95 2025-06-02 Mon 02:07:40:075.812 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241311405824, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:07:40:075.960 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241311405824, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating tag-type:deployment controller/src/ifmap/ifmap_server_table.cc 95 2025-06-02 Mon 02:07:40:118.611 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241332397824, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:07:40:118.842 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241332397824, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating tag-type:site controller/src/ifmap/ifmap_server_table.cc 95 2025-06-02 Mon 02:07:40:155.710 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241307207424, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:07:40:157.080 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241307207424, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating tag-type:neutron_fwaas controller/src/ifmap/ifmap_server_table.cc 95 2025-06-02 Mon 02:07:40:187.232 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241336596224, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:07:40:187.372 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241336596224, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating tag-type:neutron_tag controller/src/ifmap/ifmap_server_table.cc 95 2025-06-02 Mon 02:07:40:250.277 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241311405824, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:07:40:331.367 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140240543016704, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:07:40:331.537 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140240543016704, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating api-access-list:default-global-system-config:default-api-access-list controller/src/ifmap/ifmap_server_table.cc 95 2025-06-02 Mon 02:07:40:331.569 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140240543016704, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-api-access-list controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:07:40:537.120 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241294612224, Pid 1126454]: Processing scm::EvIdleHoldTimerExpired in state Idle 2025-06-02 Mon 02:07:40:537.356 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241294612224, Pid 1126454]: Connect : Start Connect timer 10.0.0.129:8086 2025-06-02 Mon 02:07:40:537.468 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241294612224, Pid 1126454]: Processing scm::EvSandeshSend in state Connect 2025-06-02 Mon 02:07:40:537.497 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241294612224, Pid 1126454]: SANDESH: Send FAILED: 1748830060537188 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-juju-os-542-1 process_status= [ [ [ module_id = contrail-dns instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.129:8086, ] ] status = Initializing description = Idle : EvIdleHoldTimerExpired -> Connect ], [ type = Database name = Cassandra server_addrs= [ [ (*_iter6) = 10.0.0.129:9041, ] ] status = Up description = Established Cassandra connection ], [ type = Database name = RabbitMQ server_addrs= [ [ (*_iter6) = 10.0.0.129:5673, ] ] status = Up description = RabbitMQ connection established ], ] ] flag_infos= [ [ ] ] description = Collector connection down ], ] ] ] 2025-06-02 Mon 02:07:40:537.506 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241294612224, Pid 1126454]: Wrong state: Connect for event: EvSandeshSend message: NodeStatusUVE 2025-06-02 Mon 02:07:40:537.514 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241294612224, Pid 1126454]: Processing scm::EvSandeshSend in state Connect 2025-06-02 Mon 02:07:40:552.842 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241294612224, Pid 1126454]: SANDESH: Send FAILED: 1748830060537454 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-juju-os-542-1:Control:contrail-dns:0 client_info= [ status = Connect successful_connections = 0 pid = 1126454 http_port = 8092 start_time = 1748830000967073 collector_name = collector_ip = 10.0.0.129:8086 collector_list= [ [ (*_iter40) = 10.0.0.129:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ConfigCassInitErrorMessage [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 4 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigClientMgrDebug [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 194 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigClientMgrWarning [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 5 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = DnsOperational [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 14 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapLinkOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 16 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapNodeOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 26 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 10 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 8 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 6 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 267, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 18, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-06-02 Mon 02:07:40:552.907 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241294612224, Pid 1126454]: Wrong state: Connect for event: EvSandeshSend message: SandeshModuleClientTrace 2025-06-02 Mon 02:07:40:706.090 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241336596224, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:07:40:706.241 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241336596224, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating application-policy-set:default-domain:default-project:default-application-policy-set controller/src/ifmap/ifmap_server_table.cc 95 2025-06-02 Mon 02:07:40:706.276 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241336596224, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating project-application-policy-set controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:07:40:853.479 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241319802624, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:07:41:404.206 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241340794624, Pid 1126454]: Processing scm::EvSandeshSend in state Connect 2025-06-02 Mon 02:07:41:404.252 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241340794624, Pid 1126454]: SANDESH: Send FAILED: 1748830061404181 [SYS_NOTICE]: UveDnsInfo: data= [ name = cn-jenkins-deploy-platform-juju-os-542-1 start_time = 1748830001402762 build_info = {"build-info":[{"build-version":"nightly","build-time":"2025-06-02 00:23:40.800365","build-user":"root","build-hostname":"jenkins-build-prepare-sandbox-9311-1.novalocal","build-id":"unknown","build-number":"unknown"}]} collector = 10.0.0.129:8086 self_ip_list= [ [ (*_iter12) = 10.0.0.129, ] ] ] 2025-06-02 Mon 02:07:41:404.272 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241340794624, Pid 1126454]: Wrong state: Connect for event: EvSandeshSend message: UveDnsInfo 2025-06-02 Mon 02:07:42:486.001 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140240543016704, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:07:42:486.406 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140240543016704, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating database-node:default-global-system-config:cn-jenkins-deploy-platform-juju-os-542-1 controller/src/ifmap/ifmap_server_table.cc 95 2025-06-02 Mon 02:07:42:486.469 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140240543016704, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-database-node controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:07:42:719.269 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241315604224, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:07:42:719.468 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241315604224, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating job-template:default-global-system-config:image_upgrade_template controller/src/ifmap/ifmap_server_table.cc 95 2025-06-02 Mon 02:07:42:719.511 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241315604224, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-job-template controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:07:42:863.929 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241303009024, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:07:43:043.931 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241311405824, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:07:43:060.067 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241303009024, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating job-template:default-global-system-config:container_cleanup_template controller/src/ifmap/ifmap_server_table.cc 95 2025-06-02 Mon 02:07:43:060.248 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241303009024, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-job-template controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:07:43:104.540 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140240543016704, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:07:43:104.716 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140240543016704, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating analytics-alarm-node:default-global-system-config:cn-jenkins-deploy-platform-juju-os-542-1 controller/src/ifmap/ifmap_server_table.cc 95 2025-06-02 Mon 02:07:43:104.750 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140240543016704, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-analytics-alarm-node controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:07:43:206.679 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241349191424, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:07:43:206.959 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241311405824, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating job-template:default-global-system-config:cli_sync_template controller/src/ifmap/ifmap_server_table.cc 95 2025-06-02 Mon 02:07:43:207.037 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241311405824, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-job-template controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:07:43:225.299 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241324001024, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:07:43:488.743 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241311405824, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:07:43:488.990 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241311405824, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating job-template:default-global-system-config:hitless_upgrade_strategy_template controller/src/ifmap/ifmap_server_table.cc 95 2025-06-02 Mon 02:07:43:489.037 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241311405824, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-job-template controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:07:43:567.539 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241290413824, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:07:43:567.769 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241290413824, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating service-appliance-set:default-global-system-config:opencontrail controller/src/ifmap/ifmap_server_table.cc 95 2025-06-02 Mon 02:07:43:567.809 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241290413824, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-service-appliance-set controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:07:43:769.109 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241298810624, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:07:43:769.315 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241298810624, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating job-template:default-global-system-config:maintenance_mode_activate_template controller/src/ifmap/ifmap_server_table.cc 95 2025-06-02 Mon 02:07:43:769.355 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241298810624, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-job-template controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:07:43:892.749 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241324001024, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:07:43:892.917 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241324001024, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating service-appliance-set:default-global-system-config:native controller/src/ifmap/ifmap_server_table.cc 95 2025-06-02 Mon 02:07:43:892.961 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241324001024, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-service-appliance-set controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:07:44:001.805 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241307207424, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:07:44:002.125 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241307207424, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating job-template:default-global-system-config:maintenance_mode_deactivate_template controller/src/ifmap/ifmap_server_table.cc 95 2025-06-02 Mon 02:07:44:002.183 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241307207424, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-job-template controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:07:44:173.783 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241332397824, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:07:44:173.968 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241332397824, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating job-template:default-global-system-config:discover_device_template controller/src/ifmap/ifmap_server_table.cc 95 2025-06-02 Mon 02:07:44:174.008 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241332397824, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-job-template controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:07:44:228.273 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241328199424, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:07:44:228.458 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241328199424, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating analytics-node:default-global-system-config:cn-jenkins-deploy-platform-juju-os-542-1 controller/src/ifmap/ifmap_server_table.cc 95 2025-06-02 Mon 02:07:44:228.503 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241328199424, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-analytics-node controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:07:44:450.869 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241307207424, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:07:44:451.060 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241307207424, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating job-template:default-global-system-config:discover_os_computes_template controller/src/ifmap/ifmap_server_table.cc 95 2025-06-02 Mon 02:07:44:451.104 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241307207424, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-job-template controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:07:44:537.741 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241303009024, Pid 1126454]: 10.0.0.129:8086 EvConnectTimerExpired in state Connect 2025-06-02 Mon 02:07:44:537.816 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241344993024, Pid 1126454]: Processing scm::EvConnectTimerExpired in state Connect 2025-06-02 Mon 02:07:44:537.852 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241344993024, Pid 1126454]: Connect : EvConnectTimerExpired 2025-06-02 Mon 02:07:44:538.044 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241344993024, Pid 1126454]: Idle 2025-06-02 Mon 02:07:44:538.160 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241344993024, Pid 1126454]: Processing scm::EvSandeshSend in state Idle 2025-06-02 Mon 02:07:44:538.189 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241344993024, Pid 1126454]: SANDESH: Send FAILED: 1748830064537892 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-juju-os-542-1 process_status= [ [ [ module_id = contrail-dns instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvConnectTimerExpired ], [ type = Database name = Cassandra server_addrs= [ [ (*_iter6) = 10.0.0.129:9041, ] ] status = Up description = Established Cassandra connection ], [ type = Database name = RabbitMQ server_addrs= [ [ (*_iter6) = 10.0.0.129:5673, ] ] status = Up description = RabbitMQ connection established ], ] ] flag_infos= [ [ ] ] description = Collector connection down ], ] ] ] 2025-06-02 Mon 02:07:44:538.197 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241344993024, Pid 1126454]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-06-02 Mon 02:07:44:538.206 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241344993024, Pid 1126454]: Processing scm::EvTcpDeleteSession in state Idle 2025-06-02 Mon 02:07:44:538.275 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241507256448, Pid 1126454]: TCP [SYS_ERR]: TcpSessionMessageLog: Session -(-1) > SSL Handshake failed due to error: 125 category: system message: Operation canceled src/contrail-common/io/ssl_server.cc 134 2025-06-02 Mon 02:07:44:538.681 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241344993024, Pid 1126454]: Processing scm::EvSandeshSend in state Idle 2025-06-02 Mon 02:07:44:539.148 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241344993024, Pid 1126454]: SANDESH: Send FAILED: 1748830064538141 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-juju-os-542-1:Control:contrail-dns:0 client_info= [ status = Idle successful_connections = 0 pid = 1126454 http_port = 8092 start_time = 1748830000967073 collector_name = collector_ip = 10.0.0.129:8086 collector_list= [ [ (*_iter40) = 10.0.0.129:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ConfigCassInitErrorMessage [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 4 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigClientMgrDebug [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 211 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigClientMgrWarning [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 5 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = DnsOperational [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 14 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapLinkOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 30 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapNodeOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 40 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 11 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 9 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 6 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveDnsInfo [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 312, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 21, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-06-02 Mon 02:07:44:539.181 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241344993024, Pid 1126454]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-06-02 Mon 02:07:44:574.895 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241349191424, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:07:44:575.103 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241344993024, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating service-template:default-domain:analyzer-template controller/src/ifmap/ifmap_server_table.cc 95 2025-06-02 Mon 02:07:44:575.163 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241344993024, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating domain-service-template controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:07:44:662.803 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241311405824, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:07:44:662.987 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241311405824, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating service-template:default-domain:nat-template controller/src/ifmap/ifmap_server_table.cc 95 2025-06-02 Mon 02:07:44:663.032 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241311405824, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating domain-service-template controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:07:44:711.274 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241324001024, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:07:44:711.489 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241324001024, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating job-template:default-global-system-config:discover_role_template controller/src/ifmap/ifmap_server_table.cc 95 2025-06-02 Mon 02:07:44:711.531 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241324001024, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-job-template controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:07:44:767.760 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241328199424, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:07:44:768.157 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241328199424, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating service-template:default-domain:netns-snat-template controller/src/ifmap/ifmap_server_table.cc 95 2025-06-02 Mon 02:07:44:768.202 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241328199424, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating domain-service-template controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:07:44:869.455 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241344993024, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:07:44:869.704 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241344993024, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating service-template:default-domain:haproxy-loadbalancer-template controller/src/ifmap/ifmap_server_table.cc 95 2025-06-02 Mon 02:07:44:869.756 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241344993024, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating domain-service-template controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:07:44:928.277 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241336596224, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:07:44:930.893 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241336596224, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating job-template:default-global-system-config:device_import_template controller/src/ifmap/ifmap_server_table.cc 95 2025-06-02 Mon 02:07:44:930.971 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241336596224, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-job-template controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:07:44:982.852 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241328199424, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:07:44:985.688 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241328199424, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating service-template:default-domain:docker-template controller/src/ifmap/ifmap_server_table.cc 95 2025-06-02 Mon 02:07:44:985.736 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241328199424, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating domain-service-template controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:07:45:124.019 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241336596224, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:07:45:124.282 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241336596224, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating job-template:default-global-system-config:fabric_config_template controller/src/ifmap/ifmap_server_table.cc 95 2025-06-02 Mon 02:07:45:124.338 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241336596224, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-job-template controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:07:45:308.689 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241303009024, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:07:45:308.914 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241328199424, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating job-template:default-global-system-config:ztp_template controller/src/ifmap/ifmap_server_table.cc 95 2025-06-02 Mon 02:07:45:308.975 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241328199424, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-job-template controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:07:45:512.139 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241324001024, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:07:45:512.318 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241324001024, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating job-template:default-global-system-config:fabric_deletion_template controller/src/ifmap/ifmap_server_table.cc 95 2025-06-02 Mon 02:07:45:512.364 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241324001024, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-job-template controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:07:45:738.490 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241349191424, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:07:45:738.782 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241349191424, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating job-template:default-global-system-config:device_deletion_template controller/src/ifmap/ifmap_server_table.cc 95 2025-06-02 Mon 02:07:45:738.838 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241349191424, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-job-template controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:07:45:943.899 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241298810624, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:07:45:944.086 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241298810624, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating job-template:default-global-system-config:role_assignment_template controller/src/ifmap/ifmap_server_table.cc 95 2025-06-02 Mon 02:07:45:944.126 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241298810624, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-job-template controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:07:46:371.719 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241294612224, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:07:46:372.433 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241294612224, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating job-template:default-global-system-config:topology_discovery_template controller/src/ifmap/ifmap_server_table.cc 95 2025-06-02 Mon 02:07:46:372.482 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241294612224, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-job-template controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:07:46:602.362 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241324001024, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:07:46:602.685 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241324001024, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating job-template:default-global-system-config:fabric_onboard_template controller/src/ifmap/ifmap_server_table.cc 95 2025-06-02 Mon 02:07:46:602.750 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241324001024, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-job-template controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:07:46:676.276 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241332397824, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:07:46:676.571 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241332397824, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating analytics-snmp-node:default-global-system-config:cn-jenkins-deploy-platform-juju-os-542-1 controller/src/ifmap/ifmap_server_table.cc 95 2025-06-02 Mon 02:07:46:676.660 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241332397824, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-analytics-snmp-node controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:07:46:808.143 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241319802624, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:07:46:808.370 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241315604224, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating job-template:default-global-system-config:search_ip_mac_template controller/src/ifmap/ifmap_server_table.cc 95 2025-06-02 Mon 02:07:46:808.436 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241315604224, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-job-template controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:07:47:006.321 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241340794624, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:07:47:006.568 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241294612224, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating job-template:default-global-system-config:show_interface_details_template controller/src/ifmap/ifmap_server_table.cc 95 2025-06-02 Mon 02:07:47:006.656 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241294612224, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-job-template controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:07:47:134.769 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241332397824, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:07:47:134.967 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241332397824, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating access-control-list:default-domain:default-project:__no_rule__:ingress-access-control-list controller/src/ifmap/ifmap_server_table.cc 95 2025-06-02 Mon 02:07:47:135.015 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241332397824, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating security-group-access-control-list controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:07:47:243.083 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241303009024, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:07:47:246.961 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241303009024, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating job-template:default-global-system-config:show_config_template controller/src/ifmap/ifmap_server_table.cc 95 2025-06-02 Mon 02:07:47:247.038 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241303009024, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-job-template controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:07:47:295.341 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140240543016704, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:07:47:295.475 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140240543016704, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating access-control-list:default-domain:default-project:__no_rule__:egress-access-control-list controller/src/ifmap/ifmap_server_table.cc 95 2025-06-02 Mon 02:07:47:295.513 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140240543016704, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating security-group-access-control-list controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:07:47:466.579 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241328199424, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:07:47:466.804 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241328199424, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating job-template:default-global-system-config:test_overlay_connectivity_template controller/src/ifmap/ifmap_server_table.cc 95 2025-06-02 Mon 02:07:47:466.849 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241328199424, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-job-template controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:07:47:470.918 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241324001024, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:07:47:506.786 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241324001024, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:07:47:507.395 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241324001024, Pid 1126454]: IFMap [SYS_WARN]: ConfigurationMalformedReferenceWarningLog: instance_target No entry in metadata map routing_instance b797f728-3795-4d3b-8b8f-c4c251d19617 controller/src/ifmap/client/config_json_parser.cc 223 2025-06-02 Mon 02:07:47:602.082 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241290413824, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:07:47:655.266 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241307207424, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:07:47:656.088 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241307207424, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating job-template:default-global-system-config:show_ops_info_template controller/src/ifmap/ifmap_server_table.cc 95 2025-06-02 Mon 02:07:47:656.139 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241307207424, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-job-template controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:07:47:689.058 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241340794624, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:07:47:689.173 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241340794624, Pid 1126454]: IFMap [SYS_WARN]: ConfigurationMalformedReferenceWarningLog: instance_target No entry in metadata map routing_instance 24543193-7b9b-485c-a256-cea4c5a59075 controller/src/ifmap/client/config_json_parser.cc 223 2025-06-02 Mon 02:07:47:789.432 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241303009024, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:07:47:869.227 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241311405824, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:07:47:869.334 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241311405824, Pid 1126454]: IFMap [SYS_WARN]: ConfigurationMalformedReferenceWarningLog: instance_target No entry in metadata map routing_instance 42148f17-2c60-42d0-b63a-897fcd809d4a controller/src/ifmap/client/config_json_parser.cc 223 2025-06-02 Mon 02:07:47:886.964 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241340794624, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:07:47:887.146 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241340794624, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating job-template:default-global-system-config:check_multicast_template controller/src/ifmap/ifmap_server_table.cc 95 2025-06-02 Mon 02:07:47:887.188 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241340794624, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-job-template controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:07:47:974.727 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241344993024, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:07:48:089.439 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241319802624, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:07:48:089.708 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241319802624, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating job-template:default-global-system-config:existing_fabric_onboard_template controller/src/ifmap/ifmap_server_table.cc 95 2025-06-02 Mon 02:07:48:089.760 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241319802624, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-job-template controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:07:48:092.290 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241344993024, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:07:48:092.394 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241344993024, Pid 1126454]: IFMap [SYS_WARN]: ConfigurationMalformedReferenceWarningLog: instance_target No entry in metadata map routing_instance 4439bfb8-6651-482b-a1c3-f5e57eb0f0f9 controller/src/ifmap/client/config_json_parser.cc 223 2025-06-02 Mon 02:07:48:233.866 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241311405824, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:07:48:235.055 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241311405824, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating job-template:default-global-system-config:vcenter_import_template controller/src/ifmap/ifmap_server_table.cc 95 2025-06-02 Mon 02:07:48:236.684 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241311405824, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-job-template controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:07:48:302.271 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241349191424, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:07:48:302.513 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241349191424, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating access-control-list:default-domain:default-project:ip-fabric:ip-fabric controller/src/ifmap/ifmap_server_table.cc 95 2025-06-02 Mon 02:07:48:302.553 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241349191424, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating virtual-network-access-control-list controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:07:48:388.578 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241294612224, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:07:48:388.923 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241294612224, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating job-template:default-global-system-config:discover_server_template controller/src/ifmap/ifmap_server_table.cc 95 2025-06-02 Mon 02:07:48:388.983 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241294612224, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-job-template controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:07:48:389.969 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241319802624, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:07:48:449.186 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241315604224, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:07:48:538.212 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241294612224, Pid 1126454]: Processing scm::EvIdleHoldTimerExpired in state Idle 2025-06-02 Mon 02:07:48:539.326 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241294612224, Pid 1126454]: Connect : Start Connect timer 10.0.0.129:8086 2025-06-02 Mon 02:07:48:539.493 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241294612224, Pid 1126454]: Processing scm::EvSandeshSend in state Connect 2025-06-02 Mon 02:07:48:539.530 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241294612224, Pid 1126454]: SANDESH: Send FAILED: 1748830068538305 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-juju-os-542-1 process_status= [ [ [ module_id = contrail-dns instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.129:8086, ] ] status = Initializing description = Idle : EvIdleHoldTimerExpired -> Connect ], [ type = Database name = Cassandra server_addrs= [ [ (*_iter6) = 10.0.0.129:9041, ] ] status = Up description = Established Cassandra connection ], [ type = Database name = RabbitMQ server_addrs= [ [ (*_iter6) = 10.0.0.129:5673, ] ] status = Up description = RabbitMQ connection established ], ] ] flag_infos= [ [ ] ] description = Collector connection down ], ] ] ] 2025-06-02 Mon 02:07:48:539.540 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241294612224, Pid 1126454]: Wrong state: Connect for event: EvSandeshSend message: NodeStatusUVE 2025-06-02 Mon 02:07:48:539.548 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241294612224, Pid 1126454]: Processing scm::EvSandeshSend in state Connect 2025-06-02 Mon 02:07:48:539.786 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241294612224, Pid 1126454]: SANDESH: Send FAILED: 1748830068539469 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-juju-os-542-1:Control:contrail-dns:0 client_info= [ status = Connect successful_connections = 0 pid = 1126454 http_port = 8092 start_time = 1748830000967073 collector_name = collector_ip = 10.0.0.129:8086 collector_list= [ [ (*_iter40) = 10.0.0.129:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ConfigCassInitErrorMessage [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 4 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigClientMgrDebug [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 248 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigClientMgrWarning [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 5 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigurationMalformedReferenceWarningLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 4 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = DnsOperational [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 14 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapLinkOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 57 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapNodeOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 67 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 12 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 10 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 7 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveDnsInfo [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 408, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 23, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-06-02 Mon 02:07:48:539.821 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241294612224, Pid 1126454]: Wrong state: Connect for event: EvSandeshSend message: SandeshModuleClientTrace 2025-06-02 Mon 02:07:48:572.318 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241294612224, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:07:48:572.502 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241294612224, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating job-template:default-global-system-config:server_import_template controller/src/ifmap/ifmap_server_table.cc 95 2025-06-02 Mon 02:07:48:572.541 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241294612224, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-job-template controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:07:48:745.382 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241290413824, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:07:48:745.631 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241290413824, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating job-template:default-global-system-config:node_profile_template controller/src/ifmap/ifmap_server_table.cc 95 2025-06-02 Mon 02:07:48:745.689 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241290413824, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-job-template controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:07:48:937.694 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241294612224, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:07:48:937.883 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241294612224, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating job-template:default-global-system-config:show_mac_mob_template controller/src/ifmap/ifmap_server_table.cc 95 2025-06-02 Mon 02:07:48:937.929 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241294612224, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-job-template controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:07:49:120.047 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241324001024, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:07:49:120.338 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241307207424, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating job-template:default-global-system-config:rma_activate_template controller/src/ifmap/ifmap_server_table.cc 95 2025-06-02 Mon 02:07:49:120.389 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241307207424, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-job-template controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:07:49:298.135 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241294612224, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:07:49:298.397 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241294612224, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating job-template:default-global-system-config:show_chassis_info_template controller/src/ifmap/ifmap_server_table.cc 95 2025-06-02 Mon 02:07:49:298.451 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241294612224, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-job-template controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:07:49:474.911 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241290413824, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:07:49:475.093 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241290413824, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating job-template:default-global-system-config:hardware_inventory_template controller/src/ifmap/ifmap_server_table.cc 95 2025-06-02 Mon 02:07:49:475.135 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241290413824, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-job-template controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:07:49:658.454 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241311405824, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:07:49:658.673 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241311405824, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating config-properties:default-global-system-config:config_property controller/src/ifmap/ifmap_server_table.cc 95 2025-06-02 Mon 02:07:49:658.716 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241311405824, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-config-properties controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:07:49:677.185 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241336596224, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:07:49:679.304 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241294612224, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating config-node:default-global-system-config:cn-jenkins-deploy-platform-juju-os-542-1 controller/src/ifmap/ifmap_server_table.cc 95 2025-06-02 Mon 02:07:49:679.514 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241294612224, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-config-node controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:07:49:840.724 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241298810624, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:07:49:840.944 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241298810624, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating tag:label=fabric-management-ip controller/src/ifmap/ifmap_server_table.cc 95 2025-06-02 Mon 02:07:49:840.997 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241298810624, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating tag-tag-type controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:07:49:984.997 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241344993024, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:07:49:985.632 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241344993024, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating tag:label=fabric-loopback-ip controller/src/ifmap/ifmap_server_table.cc 95 2025-06-02 Mon 02:07:49:985.767 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241344993024, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating tag-tag-type controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:07:50:150.722 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241324001024, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:07:50:150.973 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241303009024, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating tag:label=fabric-overlay-loopback-ip controller/src/ifmap/ifmap_server_table.cc 95 2025-06-02 Mon 02:07:50:151.033 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241303009024, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating tag-tag-type controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:07:50:301.523 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241332397824, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:07:50:304.103 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241332397824, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating tag:label=fabric-peer-ip controller/src/ifmap/ifmap_server_table.cc 95 2025-06-02 Mon 02:07:50:304.172 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241332397824, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating tag-tag-type controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:07:50:460.141 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241328199424, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:07:50:460.336 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241328199424, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating tag:label=fabric-pnf-servicechain-ip controller/src/ifmap/ifmap_server_table.cc 95 2025-06-02 Mon 02:07:50:460.374 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241328199424, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating tag-tag-type controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:07:50:599.946 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241290413824, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:07:50:600.103 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241290413824, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating tag:label=fabric-as-number controller/src/ifmap/ifmap_server_table.cc 95 2025-06-02 Mon 02:07:50:600.144 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241290413824, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating tag-tag-type controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:07:50:744.811 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241332397824, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:07:50:744.973 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241332397824, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating tag:label=fabric-ebgp-as-number controller/src/ifmap/ifmap_server_table.cc 95 2025-06-02 Mon 02:07:50:745.007 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241332397824, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating tag-tag-type controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:07:50:848.760 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241303009024, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:07:50:962.115 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241349191424, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:07:50:962.257 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241349191424, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating hardware:juniper-qfx5100-48sh-afi controller/src/ifmap/ifmap_server_table.cc 95 2025-06-02 Mon 02:07:51:082.334 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241294612224, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:07:51:082.668 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241294612224, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating hardware:juniper-qfx5100-48sh-afo controller/src/ifmap/ifmap_server_table.cc 95 2025-06-02 Mon 02:07:51:230.038 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140240543016704, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:07:51:230.192 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140240543016704, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating hardware:juniper-qfx5100-48th-afi controller/src/ifmap/ifmap_server_table.cc 95 2025-06-02 Mon 02:07:51:336.320 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241307207424, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:07:51:336.459 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241307207424, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating hardware:juniper-qfx5100-48th-afo controller/src/ifmap/ifmap_server_table.cc 95 2025-06-02 Mon 02:07:51:397.609 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241340794624, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:07:51:397.746 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241340794624, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating config-database-node:default-global-system-config:cn-jenkins-deploy-platform-juju-os-542-1 controller/src/ifmap/ifmap_server_table.cc 95 2025-06-02 Mon 02:07:51:397.779 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241340794624, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-config-database-node controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:07:51:470.203 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241336596224, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:07:51:470.435 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241336596224, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating hardware:juniper-qfx5100-48s-6q controller/src/ifmap/ifmap_server_table.cc 95 2025-06-02 Mon 02:07:51:588.377 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241336596224, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:07:51:588.511 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241336596224, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating hardware:juniper-qfx5100-96s-8q controller/src/ifmap/ifmap_server_table.cc 95 2025-06-02 Mon 02:07:51:701.540 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241298810624, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:07:51:701.689 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241298810624, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating hardware:juniper-qfx5100-48t controller/src/ifmap/ifmap_server_table.cc 95 2025-06-02 Mon 02:07:51:821.414 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241307207424, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:07:51:821.782 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241307207424, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating hardware:juniper-qfx5100-48t-6q controller/src/ifmap/ifmap_server_table.cc 95 2025-06-02 Mon 02:07:51:942.779 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241307207424, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:07:51:942.897 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241307207424, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating hardware:juniper-qfx5100-24q-2p controller/src/ifmap/ifmap_server_table.cc 95 2025-06-02 Mon 02:07:52:051.090 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241303009024, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:07:52:051.243 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241303009024, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating hardware:juniper-qfx5100-24q-aa controller/src/ifmap/ifmap_server_table.cc 95 2025-06-02 Mon 02:07:52:167.429 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241344993024, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:07:52:167.582 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241344993024, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating hardware:juniper-qfx5100e-48s-6q controller/src/ifmap/ifmap_server_table.cc 95 2025-06-02 Mon 02:07:52:281.172 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241290413824, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:07:52:281.291 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241290413824, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating hardware:juniper-qfx5100e-96s-8q controller/src/ifmap/ifmap_server_table.cc 95 2025-06-02 Mon 02:07:52:393.652 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241315604224, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:07:52:393.793 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241315604224, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating hardware:juniper-qfx5100e-48t-6q controller/src/ifmap/ifmap_server_table.cc 95 2025-06-02 Mon 02:07:52:509.238 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241319802624, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:07:52:509.370 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241319802624, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating hardware:juniper-qfx5100e-24q-2p controller/src/ifmap/ifmap_server_table.cc 95 2025-06-02 Mon 02:07:52:619.106 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241298810624, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:07:52:619.410 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241298810624, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating hardware:juniper-qfx5110-48s-4c controller/src/ifmap/ifmap_server_table.cc 95 2025-06-02 Mon 02:07:52:732.719 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241315604224, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:07:52:733.279 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241315604224, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating hardware:juniper-qfx5110-32q controller/src/ifmap/ifmap_server_table.cc 95 2025-06-02 Mon 02:07:52:850.315 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241307207424, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:07:52:850.477 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241307207424, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating hardware:juniper-qfx5120-48y-8c controller/src/ifmap/ifmap_server_table.cc 95 2025-06-02 Mon 02:07:52:965.009 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241303009024, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:07:52:965.139 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241303009024, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating hardware:juniper-qfx5120-32c controller/src/ifmap/ifmap_server_table.cc 95 2025-06-02 Mon 02:07:53:085.713 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241298810624, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:07:53:085.904 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241294612224, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating hardware:juniper-qfx5120-48t-6c controller/src/ifmap/ifmap_server_table.cc 95 2025-06-02 Mon 02:07:53:194.548 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241298810624, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:07:53:194.688 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241298810624, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating hardware:juniper-qfx5220-32cd controller/src/ifmap/ifmap_server_table.cc 95 2025-06-02 Mon 02:07:53:301.982 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241319802624, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:07:53:302.187 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241324001024, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating hardware:juniper-qfx5220-128c controller/src/ifmap/ifmap_server_table.cc 95 2025-06-02 Mon 02:07:53:423.004 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241298810624, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:07:53:423.158 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241298810624, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating hardware:juniper-qfx5200-32c-32q controller/src/ifmap/ifmap_server_table.cc 95 2025-06-02 Mon 02:07:53:537.686 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241315604224, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:07:53:537.900 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241328199424, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating hardware:juniper-qfx5200-48y controller/src/ifmap/ifmap_server_table.cc 95 2025-06-02 Mon 02:07:53:666.992 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241294612224, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:07:53:667.161 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241294612224, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating hardware:juniper-qfx5210-64c controller/src/ifmap/ifmap_server_table.cc 95 2025-06-02 Mon 02:07:53:782.329 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241328199424, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:07:53:782.465 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241328199424, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating hardware:juniper-qfx10002-72q controller/src/ifmap/ifmap_server_table.cc 95 2025-06-02 Mon 02:07:53:895.444 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241319802624, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:07:53:895.573 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241319802624, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating hardware:juniper-qfx10002-36q controller/src/ifmap/ifmap_server_table.cc 95 2025-06-02 Mon 02:07:54:021.246 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241340794624, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:07:54:021.385 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241340794624, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating hardware:juniper-qfx10002-60c controller/src/ifmap/ifmap_server_table.cc 95 2025-06-02 Mon 02:07:54:127.943 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241311405824, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:07:54:128.106 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241311405824, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating hardware:juniper-qfx10016 controller/src/ifmap/ifmap_server_table.cc 95 2025-06-02 Mon 02:07:54:244.842 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241307207424, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:07:54:244.999 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241307207424, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating hardware:juniper-qfx10008 controller/src/ifmap/ifmap_server_table.cc 95 2025-06-02 Mon 02:07:54:364.321 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241336596224, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:07:54:364.466 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241336596224, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating hardware:juniper-vqfx-10000 controller/src/ifmap/ifmap_server_table.cc 95 2025-06-02 Mon 02:07:54:475.017 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241290413824, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:07:54:475.175 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241290413824, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating hardware:juniper-mx80 controller/src/ifmap/ifmap_server_table.cc 95 2025-06-02 Mon 02:07:54:587.962 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241311405824, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:07:54:588.078 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241311405824, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating hardware:juniper-mx80-48t controller/src/ifmap/ifmap_server_table.cc 95 2025-06-02 Mon 02:07:54:708.823 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241349191424, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:07:54:708.986 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241344993024, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating hardware:juniper-mx204 controller/src/ifmap/ifmap_server_table.cc 95 2025-06-02 Mon 02:07:54:817.622 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241344993024, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:07:54:817.745 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241344993024, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating hardware:juniper-mx240 controller/src/ifmap/ifmap_server_table.cc 95 2025-06-02 Mon 02:07:54:924.758 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241311405824, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:07:54:924.914 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241311405824, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating hardware:juniper-mx480 controller/src/ifmap/ifmap_server_table.cc 95 2025-06-02 Mon 02:07:55:043.292 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241324001024, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:07:55:043.417 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241328199424, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating hardware:juniper-mx960 controller/src/ifmap/ifmap_server_table.cc 95 2025-06-02 Mon 02:07:55:165.879 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241303009024, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:07:55:166.021 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241303009024, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating hardware:juniper-mx2008 controller/src/ifmap/ifmap_server_table.cc 95 2025-06-02 Mon 02:07:55:279.194 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241311405824, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:07:55:279.924 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241298810624, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating hardware:juniper-mx2010 controller/src/ifmap/ifmap_server_table.cc 95 2025-06-02 Mon 02:07:55:391.901 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241332397824, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:07:55:392.079 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241340794624, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating hardware:juniper-mx2020 controller/src/ifmap/ifmap_server_table.cc 95 2025-06-02 Mon 02:07:55:499.559 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241294612224, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:07:55:499.693 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241294612224, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating hardware:juniper-mx10003 controller/src/ifmap/ifmap_server_table.cc 95 2025-06-02 Mon 02:07:55:617.188 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241344993024, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:07:55:617.377 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241344993024, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating hardware:juniper-jnp10008 controller/src/ifmap/ifmap_server_table.cc 95 2025-06-02 Mon 02:07:55:721.588 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241336596224, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:07:55:721.776 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241336596224, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating hardware:juniper-jnp10016 controller/src/ifmap/ifmap_server_table.cc 95 2025-06-02 Mon 02:07:55:820.672 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241328199424, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:07:55:853.084 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241290413824, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:07:55:853.206 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241290413824, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating hardware:juniper-vmx controller/src/ifmap/ifmap_server_table.cc 95 2025-06-02 Mon 02:07:55:968.900 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241307207424, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:07:55:969.041 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241315604224, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating hardware:juniper-srx5400 controller/src/ifmap/ifmap_server_table.cc 95 2025-06-02 Mon 02:07:56:089.627 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241328199424, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:07:56:089.813 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241319802624, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating hardware:juniper-srx5600 controller/src/ifmap/ifmap_server_table.cc 95 2025-06-02 Mon 02:07:56:203.160 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241303009024, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:07:56:203.327 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241328199424, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating hardware:juniper-srx4600 controller/src/ifmap/ifmap_server_table.cc 95 2025-06-02 Mon 02:07:56:323.508 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241315604224, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:07:56:323.653 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241315604224, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating hardware:juniper-srx4100 controller/src/ifmap/ifmap_server_table.cc 95 2025-06-02 Mon 02:07:56:438.522 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241290413824, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:07:56:438.688 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241290413824, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating hardware:juniper-srx1500 controller/src/ifmap/ifmap_server_table.cc 95 2025-06-02 Mon 02:07:56:539.473 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241311405824, Pid 1126454]: 10.0.0.129:8086 EvConnectTimerExpired in state Connect 2025-06-02 Mon 02:07:56:541.934 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241311405824, Pid 1126454]: Processing scm::EvConnectTimerExpired in state Connect 2025-06-02 Mon 02:07:56:541.963 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241311405824, Pid 1126454]: Connect : EvConnectTimerExpired 2025-06-02 Mon 02:07:56:542.106 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241507256448, Pid 1126454]: TCP [SYS_ERR]: TcpSessionMessageLog: Session -(-1) > SSL Handshake failed due to error: 335544539 category: asio.ssl message: short read src/contrail-common/io/ssl_server.cc 134 2025-06-02 Mon 02:07:56:542.136 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241311405824, Pid 1126454]: Idle 2025-06-02 Mon 02:07:56:542.245 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241311405824, Pid 1126454]: Processing scm::EvSandeshSend in state Idle 2025-06-02 Mon 02:07:56:542.275 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241311405824, Pid 1126454]: SANDESH: Send FAILED: 1748830076542007 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-juju-os-542-1 process_status= [ [ [ module_id = contrail-dns instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvConnectTimerExpired ], [ type = Database name = Cassandra server_addrs= [ [ (*_iter6) = 10.0.0.129:9041, ] ] status = Up description = Established Cassandra connection ], [ type = Database name = RabbitMQ server_addrs= [ [ (*_iter6) = 10.0.0.129:5673, ] ] status = Up description = RabbitMQ connection established ], ] ] flag_infos= [ [ ] ] description = Collector connection down ], ] ] ] 2025-06-02 Mon 02:07:56:542.283 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241311405824, Pid 1126454]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-06-02 Mon 02:07:56:542.291 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241311405824, Pid 1126454]: Processing scm::EvTcpDeleteSession in state Idle 2025-06-02 Mon 02:07:56:542.321 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241311405824, Pid 1126454]: Processing scm::EvSandeshSend in state Idle 2025-06-02 Mon 02:07:56:542.533 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241311405824, Pid 1126454]: SANDESH: Send FAILED: 1748830076542230 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-juju-os-542-1:Control:contrail-dns:0 client_info= [ status = Idle successful_connections = 0 pid = 1126454 http_port = 8092 start_time = 1748830000967073 collector_name = collector_ip = 10.0.0.129:8086 collector_list= [ [ (*_iter40) = 10.0.0.129:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ConfigCassInitErrorMessage [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 4 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigClientMgrDebug [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 314 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigClientMgrWarning [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 5 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigurationMalformedReferenceWarningLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 4 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = DnsOperational [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 14 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapLinkOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 73 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapNodeOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 131 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 13 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 11 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 8 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveDnsInfo [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 555, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 25, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-06-02 Mon 02:07:56:542.559 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241311405824, Pid 1126454]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-06-02 Mon 02:07:56:548.823 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241311405824, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:07:56:548.993 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241311405824, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating hardware:juniper-srx240h-poe controller/src/ifmap/ifmap_server_table.cc 95 2025-06-02 Mon 02:07:56:671.906 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241303009024, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:07:56:672.171 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241303009024, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating hardware:juniper-srx5800 controller/src/ifmap/ifmap_server_table.cc 95 2025-06-02 Mon 02:07:56:722.746 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241315604224, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:07:56:722.944 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241303009024, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating alarm:default-global-system-config:system-defined-conf-elements-inconsistency controller/src/ifmap/ifmap_server_table.cc 95 2025-06-02 Mon 02:07:56:723.165 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241303009024, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-alarm controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:07:56:778.373 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241336596224, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:07:56:778.540 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241336596224, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating alarm:default-global-system-config:system-defined-address-mismatch-compute controller/src/ifmap/ifmap_server_table.cc 95 2025-06-02 Mon 02:07:56:778.588 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241336596224, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-alarm controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:07:56:790.164 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241332397824, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:07:56:790.299 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241332397824, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating hardware:juniper-srx4200 controller/src/ifmap/ifmap_server_table.cc 95 2025-06-02 Mon 02:07:56:854.221 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140240543016704, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:07:56:854.394 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241315604224, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating alarm:default-global-system-config:system-defined-address-mismatch-control controller/src/ifmap/ifmap_server_table.cc 95 2025-06-02 Mon 02:07:56:854.448 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241315604224, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-alarm controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:07:57:002.825 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241303009024, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:07:57:002.969 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241303009024, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating hardware:juniper-vsrx controller/src/ifmap/ifmap_server_table.cc 95 2025-06-02 Mon 02:07:57:015.319 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241349191424, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:07:57:015.472 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241349191424, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating alarm:default-global-system-config:system-defined-bgp-connectivity controller/src/ifmap/ifmap_server_table.cc 95 2025-06-02 Mon 02:07:57:015.509 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241349191424, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-alarm controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:07:57:098.259 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241311405824, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:07:57:102.330 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241311405824, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating alarm:default-global-system-config:system-defined-conf-incorrect controller/src/ifmap/ifmap_server_table.cc 95 2025-06-02 Mon 02:07:57:102.389 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241311405824, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-alarm controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:07:57:145.963 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241340794624, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:07:57:146.152 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241340794624, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating feature:default-global-system-config:underlay-ip-clos controller/src/ifmap/ifmap_server_table.cc 95 2025-06-02 Mon 02:07:57:146.191 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241340794624, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:07:57:177.155 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241303009024, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:07:57:177.370 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241303009024, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating alarm:default-global-system-config:system-defined-disk-usage-high controller/src/ifmap/ifmap_server_table.cc 95 2025-06-02 Mon 02:07:57:177.411 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241303009024, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-alarm controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:07:57:252.412 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241303009024, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:07:57:252.609 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241303009024, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating alarm:default-global-system-config:system-defined-disk-usage-critical controller/src/ifmap/ifmap_server_table.cc 95 2025-06-02 Mon 02:07:57:252.660 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241303009024, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-alarm controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:07:57:310.268 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241328199424, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:07:57:310.914 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241328199424, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating feature:default-global-system-config:overlay-bgp controller/src/ifmap/ifmap_server_table.cc 95 2025-06-02 Mon 02:07:57:310.976 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241328199424, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:07:57:353.253 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241307207424, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:07:57:358.041 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241344993024, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating alarm:default-global-system-config:system-defined-node-status controller/src/ifmap/ifmap_server_table.cc 95 2025-06-02 Mon 02:07:57:358.136 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241344993024, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-alarm controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:07:57:434.232 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241294612224, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:07:57:434.392 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241294612224, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating feature:default-global-system-config:l2-gateway controller/src/ifmap/ifmap_server_table.cc 95 2025-06-02 Mon 02:07:57:434.433 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241294612224, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:07:57:456.170 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241311405824, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:07:57:456.349 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241311405824, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating alarm:default-global-system-config:system-defined-partial-sysinfo controller/src/ifmap/ifmap_server_table.cc 95 2025-06-02 Mon 02:07:57:456.385 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241311405824, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-alarm controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:07:57:520.175 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241303009024, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:07:57:520.415 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241303009024, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating alarm:default-global-system-config:system-defined-process-connectivity controller/src/ifmap/ifmap_server_table.cc 95 2025-06-02 Mon 02:07:57:520.456 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241303009024, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-alarm controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:07:57:570.546 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140240543016704, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:07:57:570.726 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140240543016704, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating feature:default-global-system-config:l3-gateway controller/src/ifmap/ifmap_server_table.cc 95 2025-06-02 Mon 02:07:57:570.770 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140240543016704, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:07:57:599.498 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241344993024, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:07:57:600.117 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241324001024, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating alarm:default-global-system-config:system-defined-process-status controller/src/ifmap/ifmap_server_table.cc 95 2025-06-02 Mon 02:07:57:600.191 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241324001024, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-alarm controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:07:57:680.627 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241311405824, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:07:57:680.817 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241311405824, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating alarm:default-global-system-config:system-defined-prouter-connectivity controller/src/ifmap/ifmap_server_table.cc 95 2025-06-02 Mon 02:07:57:680.861 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241311405824, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-alarm controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:07:57:706.318 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241319802624, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:07:57:706.475 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241319802624, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating feature:default-global-system-config:vn-interconnect controller/src/ifmap/ifmap_server_table.cc 95 2025-06-02 Mon 02:07:57:706.510 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241319802624, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:07:57:807.702 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241303009024, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:07:57:807.889 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241303009024, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating alarm:default-global-system-config:system-defined-prouter-tsn-connectivity controller/src/ifmap/ifmap_server_table.cc 95 2025-06-02 Mon 02:07:57:807.927 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241303009024, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-alarm controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:07:57:841.824 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140240543016704, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:07:57:841.958 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140240543016704, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating feature:default-global-system-config:dc-gateway controller/src/ifmap/ifmap_server_table.cc 95 2025-06-02 Mon 02:07:57:842.781 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140240543016704, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:07:57:891.379 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241344993024, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:07:57:891.626 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241336596224, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating alarm:default-global-system-config:system-defined-storage-cluster-state controller/src/ifmap/ifmap_server_table.cc 95 2025-06-02 Mon 02:07:57:891.684 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241336596224, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-alarm controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:07:57:960.750 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241311405824, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:07:57:961.043 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241349191424, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating alarm:default-global-system-config:system-defined-vrouter-interface controller/src/ifmap/ifmap_server_table.cc 95 2025-06-02 Mon 02:07:57:961.103 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241349191424, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-alarm controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:07:57:988.855 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241315604224, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:07:57:989.099 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241315604224, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating feature:default-global-system-config:assisted-replicator controller/src/ifmap/ifmap_server_table.cc 95 2025-06-02 Mon 02:07:57:989.135 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241315604224, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:07:58:045.426 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241328199424, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:07:58:045.621 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241328199424, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating alarm:default-global-system-config:system-defined-xmpp-connectivity controller/src/ifmap/ifmap_server_table.cc 95 2025-06-02 Mon 02:07:58:045.672 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241328199424, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-alarm controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:07:58:133.371 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241298810624, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:07:58:133.564 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241298810624, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating feature:default-global-system-config:port-profile controller/src/ifmap/ifmap_server_table.cc 95 2025-06-02 Mon 02:07:58:133.643 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241298810624, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:07:58:150.330 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241344993024, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:07:58:150.530 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241344993024, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating alarm:default-global-system-config:system-defined-xmpp-close-reason controller/src/ifmap/ifmap_server_table.cc 95 2025-06-02 Mon 02:07:58:150.572 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241344993024, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-alarm controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:07:58:242.134 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241315604224, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:07:58:242.314 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241315604224, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating alarm:default-global-system-config:system-defined-core-files controller/src/ifmap/ifmap_server_table.cc 95 2025-06-02 Mon 02:07:58:242.351 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241315604224, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-alarm controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:07:58:262.237 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241298810624, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:07:58:262.383 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241298810624, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating feature:default-global-system-config:telemetry controller/src/ifmap/ifmap_server_table.cc 95 2025-06-02 Mon 02:07:58:262.418 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241298810624, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:07:58:337.698 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241340794624, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:07:58:337.962 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241340794624, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating alarm:default-global-system-config:system-defined-pending-cassandra-compaction-tasks controller/src/ifmap/ifmap_server_table.cc 95 2025-06-02 Mon 02:07:58:338.007 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241340794624, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-alarm controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:07:58:396.384 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241311405824, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:07:58:397.185 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241311405824, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating feature:default-global-system-config:firewall controller/src/ifmap/ifmap_server_table.cc 95 2025-06-02 Mon 02:07:58:397.260 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241311405824, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:07:58:431.921 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241319802624, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:07:58:432.090 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241319802624, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating alarm:default-global-system-config:system-defined-package-version-mismatch controller/src/ifmap/ifmap_server_table.cc 95 2025-06-02 Mon 02:07:58:432.128 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241319802624, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-alarm controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:07:58:506.317 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241324001024, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:07:58:507.048 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241324001024, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating alarm:default-global-system-config:system-defined-vrouter-limit-exceeded controller/src/ifmap/ifmap_server_table.cc 95 2025-06-02 Mon 02:07:58:507.110 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241324001024, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-alarm controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:07:58:547.699 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241311405824, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:07:58:547.850 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241311405824, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating feature:default-global-system-config:infra-bms-access controller/src/ifmap/ifmap_server_table.cc 95 2025-06-02 Mon 02:07:58:547.885 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241311405824, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:07:58:573.712 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241303009024, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:07:58:573.878 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241303009024, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating alarm:default-global-system-config:system-defined-vrouter-table-limit-exceeded controller/src/ifmap/ifmap_server_table.cc 95 2025-06-02 Mon 02:07:58:573.914 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241303009024, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-alarm controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:07:58:661.547 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241324001024, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:07:58:661.760 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241324001024, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating alarm:default-global-system-config:system-defined-vrouter-l3mh-physical-interface-link-down controller/src/ifmap/ifmap_server_table.cc 95 2025-06-02 Mon 02:07:58:662.392 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241324001024, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-alarm controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:07:58:687.032 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241340794624, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:07:58:687.166 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241340794624, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating feature:default-global-system-config:pnf-service-chaining controller/src/ifmap/ifmap_server_table.cc 95 2025-06-02 Mon 02:07:58:687.208 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241340794624, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:07:58:821.886 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241344993024, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:07:58:822.068 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241344993024, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating physical-role:default-global-system-config:leaf controller/src/ifmap/ifmap_server_table.cc 95 2025-06-02 Mon 02:07:58:822.244 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241344993024, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-physical-role controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:07:58:937.639 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241311405824, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:07:58:937.819 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241311405824, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating physical-role:default-global-system-config:spine controller/src/ifmap/ifmap_server_table.cc 95 2025-06-02 Mon 02:07:58:937.862 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241311405824, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-physical-role controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:07:59:068.890 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241303009024, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:07:59:069.058 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241303009024, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating physical-role:default-global-system-config:superspine controller/src/ifmap/ifmap_server_table.cc 95 2025-06-02 Mon 02:07:59:069.104 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241303009024, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-physical-role controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:07:59:204.034 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241340794624, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:07:59:204.217 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241298810624, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating physical-role:default-global-system-config:pnf controller/src/ifmap/ifmap_server_table.cc 95 2025-06-02 Mon 02:07:59:204.280 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241298810624, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-physical-role controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:07:59:343.068 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241315604224, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:07:59:343.393 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241315604224, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating overlay-role:default-global-system-config:erb-ucast-gateway controller/src/ifmap/ifmap_server_table.cc 95 2025-06-02 Mon 02:07:59:343.437 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241315604224, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-overlay-role controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:07:59:533.659 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241324001024, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:07:59:533.844 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241324001024, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating overlay-role:default-global-system-config:crb-access controller/src/ifmap/ifmap_server_table.cc 95 2025-06-02 Mon 02:07:59:533.894 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241324001024, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-overlay-role controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:07:59:578.932 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241290413824, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:07:59:764.897 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241319802624, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:07:59:765.391 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241311405824, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating overlay-role:default-global-system-config:crb-mcast-gateway controller/src/ifmap/ifmap_server_table.cc 95 2025-06-02 Mon 02:07:59:765.451 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241311405824, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-overlay-role controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:07:59:907.990 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241294612224, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:07:59:919.746 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241294612224, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating overlay-role:default-global-system-config:crb-gateway controller/src/ifmap/ifmap_server_table.cc 95 2025-06-02 Mon 02:07:59:919.803 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241294612224, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-overlay-role controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:08:00:057.336 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241344993024, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:08:00:057.518 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241344993024, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating overlay-role:default-global-system-config:collapsed-spine controller/src/ifmap/ifmap_server_table.cc 95 2025-06-02 Mon 02:08:00:057.578 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241344993024, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-overlay-role controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:08:00:174.516 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241340794624, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:08:00:307.142 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241311405824, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:08:00:307.281 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241311405824, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating overlay-role:default-global-system-config:dc-gateway controller/src/ifmap/ifmap_server_table.cc 95 2025-06-02 Mon 02:08:00:307.318 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241311405824, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-overlay-role controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:08:00:450.502 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241307207424, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:08:00:450.819 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241307207424, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating overlay-role:default-global-system-config:dci-gateway controller/src/ifmap/ifmap_server_table.cc 95 2025-06-02 Mon 02:08:00:450.964 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241307207424, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-overlay-role controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:08:00:591.272 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241311405824, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:08:00:591.483 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241328199424, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating overlay-role:default-global-system-config:ar-client controller/src/ifmap/ifmap_server_table.cc 95 2025-06-02 Mon 02:08:00:591.542 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241328199424, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-overlay-role controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:08:00:600.641 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241344993024, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:08:00:724.330 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241298810624, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:08:00:724.700 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241340794624, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating overlay-role:default-global-system-config:ar-replicator controller/src/ifmap/ifmap_server_table.cc 95 2025-06-02 Mon 02:08:00:724.757 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241340794624, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-overlay-role controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:08:00:852.813 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241298810624, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:08:00:853.328 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241298810624, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating overlay-role:default-global-system-config:route-reflector controller/src/ifmap/ifmap_server_table.cc 95 2025-06-02 Mon 02:08:00:853.373 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241298810624, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-overlay-role controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:08:00:994.187 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241328199424, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:08:00:994.366 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241328199424, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating overlay-role:default-global-system-config:lean controller/src/ifmap/ifmap_server_table.cc 95 2025-06-02 Mon 02:08:00:994.411 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241328199424, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-overlay-role controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:08:01:138.103 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241340794624, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:08:01:138.261 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241340794624, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating overlay-role:default-global-system-config:pnf-servicechain controller/src/ifmap/ifmap_server_table.cc 95 2025-06-02 Mon 02:08:01:138.296 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241340794624, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-overlay-role controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:08:01:264.973 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241324001024, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:08:01:265.130 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241303009024, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating role-definition:default-global-system-config:erb-leaf controller/src/ifmap/ifmap_server_table.cc 95 2025-06-02 Mon 02:08:01:265.193 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241303009024, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-role-definition controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:08:01:411.839 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241344993024, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:08:01:412.028 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241340794624, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating role-definition:default-global-system-config:crb-access-leaf controller/src/ifmap/ifmap_server_table.cc 95 2025-06-02 Mon 02:08:01:412.155 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241340794624, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-role-definition controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:08:01:557.299 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241344993024, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:08:01:557.466 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241344993024, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating role-definition:default-global-system-config:crb-access-spine controller/src/ifmap/ifmap_server_table.cc 95 2025-06-02 Mon 02:08:01:557.500 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241344993024, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-role-definition controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:08:01:687.481 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241332397824, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:08:01:687.884 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241332397824, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating role-definition:default-global-system-config:crb-gateway-spine controller/src/ifmap/ifmap_server_table.cc 95 2025-06-02 Mon 02:08:01:687.945 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241332397824, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-role-definition controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:08:01:814.155 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241311405824, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:08:01:841.393 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241303009024, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:08:01:841.748 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241303009024, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating role-definition:default-global-system-config:crb-mcast-gateway-spine controller/src/ifmap/ifmap_server_table.cc 95 2025-06-02 Mon 02:08:01:841.796 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241303009024, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-role-definition controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:08:01:971.010 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241315604224, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:08:01:971.139 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241315604224, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating role-definition:default-global-system-config:collapsed-spine controller/src/ifmap/ifmap_server_table.cc 95 2025-06-02 Mon 02:08:01:971.173 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241315604224, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-role-definition controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:08:02:107.025 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241290413824, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:08:02:107.193 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241328199424, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating role-definition:default-global-system-config:dc-gateway-spine controller/src/ifmap/ifmap_server_table.cc 95 2025-06-02 Mon 02:08:02:107.865 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241328199424, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-role-definition controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:08:02:251.802 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241349191424, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:08:02:251.961 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241349191424, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating role-definition:default-global-system-config:dci-gateway-spine controller/src/ifmap/ifmap_server_table.cc 95 2025-06-02 Mon 02:08:02:251.997 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241349191424, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-role-definition controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:08:02:372.622 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241344993024, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:08:02:372.998 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241324001024, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating role-definition:default-global-system-config:crb-gateway-leaf controller/src/ifmap/ifmap_server_table.cc 95 2025-06-02 Mon 02:08:02:373.063 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241324001024, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-role-definition controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:08:02:497.340 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241294612224, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:08:02:497.481 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241294612224, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating role-definition:default-global-system-config:crb-mcast-gateway-leaf controller/src/ifmap/ifmap_server_table.cc 95 2025-06-02 Mon 02:08:02:497.524 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241294612224, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-role-definition controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:08:02:637.172 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241303009024, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:08:02:637.302 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241303009024, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating role-definition:default-global-system-config:dc-gateway-leaf controller/src/ifmap/ifmap_server_table.cc 95 2025-06-02 Mon 02:08:02:637.337 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241303009024, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-role-definition controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:08:02:758.793 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241336596224, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:08:02:759.023 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241332397824, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating role-definition:default-global-system-config:dci-gateway-leaf controller/src/ifmap/ifmap_server_table.cc 95 2025-06-02 Mon 02:08:02:760.408 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241332397824, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-role-definition controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:08:02:880.265 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241336596224, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:08:02:880.412 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241336596224, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating role-definition:default-global-system-config:ar-client-leaf controller/src/ifmap/ifmap_server_table.cc 95 2025-06-02 Mon 02:08:02:880.448 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241336596224, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-role-definition controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:08:03:020.258 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241303009024, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:08:03:020.482 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241336596224, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating role-definition:default-global-system-config:ar-client-spine controller/src/ifmap/ifmap_server_table.cc 95 2025-06-02 Mon 02:08:03:020.641 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241336596224, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-role-definition controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:08:03:151.588 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241319802624, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:08:03:151.754 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241319802624, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating role-definition:default-global-system-config:ar-replicator-leaf controller/src/ifmap/ifmap_server_table.cc 95 2025-06-02 Mon 02:08:03:151.789 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241319802624, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-role-definition controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:08:03:273.549 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241340794624, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:08:03:273.702 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241340794624, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating role-definition:default-global-system-config:ar-replicator-spine controller/src/ifmap/ifmap_server_table.cc 95 2025-06-02 Mon 02:08:03:273.740 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241340794624, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-role-definition controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:08:03:401.875 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241332397824, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:08:03:402.030 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241332397824, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating role-definition:default-global-system-config:rr-spine controller/src/ifmap/ifmap_server_table.cc 95 2025-06-02 Mon 02:08:03:402.067 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241332397824, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-role-definition controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:08:03:532.937 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241298810624, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:08:03:533.116 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241298810624, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating role-definition:default-global-system-config:rr-superspine controller/src/ifmap/ifmap_server_table.cc 95 2025-06-02 Mon 02:08:03:533.156 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241298810624, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-role-definition controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:08:03:663.739 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241324001024, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:08:03:663.904 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241324001024, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating role-definition:default-global-system-config:rr-leaf controller/src/ifmap/ifmap_server_table.cc 95 2025-06-02 Mon 02:08:03:663.952 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241324001024, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-role-definition controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:08:03:795.465 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241298810624, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:08:03:795.654 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241298810624, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating role-definition:default-global-system-config:lean-spine controller/src/ifmap/ifmap_server_table.cc 95 2025-06-02 Mon 02:08:03:795.709 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241298810624, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-role-definition controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:08:03:934.858 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241344993024, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:08:03:935.231 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241336596224, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating role-definition:default-global-system-config:lean-superspine controller/src/ifmap/ifmap_server_table.cc 95 2025-06-02 Mon 02:08:03:935.288 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241336596224, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-role-definition controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:08:04:061.637 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241303009024, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:08:04:061.858 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241303009024, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating role-definition:default-global-system-config:pnf-servicechain-pnf controller/src/ifmap/ifmap_server_table.cc 95 2025-06-02 Mon 02:08:04:061.907 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241303009024, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-role-definition controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:08:04:192.705 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140240543016704, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:08:04:192.942 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140240543016704, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating role-definition:default-global-system-config:pnf-servicechain-spine controller/src/ifmap/ifmap_server_table.cc 95 2025-06-02 Mon 02:08:04:192.983 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140240543016704, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-role-definition controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:08:04:338.666 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140240543016704, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:08:04:338.812 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140240543016704, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating role-definition:default-global-system-config:pnf-servicechain-leaf controller/src/ifmap/ifmap_server_table.cc 95 2025-06-02 Mon 02:08:04:338.855 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140240543016704, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-role-definition controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:08:04:466.824 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140240543016704, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:08:04:466.978 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140240543016704, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating feature-config:default-global-system-config:erb-leaf:l3-gateway controller/src/ifmap/ifmap_server_table.cc 95 2025-06-02 Mon 02:08:04:467.016 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140240543016704, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature-config controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:08:04:542.304 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241298810624, Pid 1126454]: Processing scm::EvIdleHoldTimerExpired in state Idle 2025-06-02 Mon 02:08:04:542.917 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241298810624, Pid 1126454]: Connect : Start Connect timer 10.0.0.129:8086 2025-06-02 Mon 02:08:04:543.072 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241298810624, Pid 1126454]: Processing scm::EvSandeshSend in state Connect 2025-06-02 Mon 02:08:04:544.697 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241298810624, Pid 1126454]: SANDESH: Send FAILED: 1748830084542378 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-juju-os-542-1 process_status= [ [ [ module_id = contrail-dns instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.129:8086, ] ] status = Initializing description = Idle : EvIdleHoldTimerExpired -> Connect ], [ type = Database name = Cassandra server_addrs= [ [ (*_iter6) = 10.0.0.129:9041, ] ] status = Up description = Established Cassandra connection ], [ type = Database name = RabbitMQ server_addrs= [ [ (*_iter6) = 10.0.0.129:5673, ] ] status = Up description = RabbitMQ connection established ], ] ] flag_infos= [ [ ] ] description = Collector connection down ], ] ] ] 2025-06-02 Mon 02:08:04:544.726 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241298810624, Pid 1126454]: Wrong state: Connect for event: EvSandeshSend message: NodeStatusUVE 2025-06-02 Mon 02:08:04:544.741 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241298810624, Pid 1126454]: Processing scm::EvSandeshSend in state Connect 2025-06-02 Mon 02:08:04:545.227 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241298810624, Pid 1126454]: SANDESH: Send FAILED: 1748830084543045 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-juju-os-542-1:Control:contrail-dns:0 client_info= [ status = Connect successful_connections = 0 pid = 1126454 http_port = 8092 start_time = 1748830000967073 collector_name = collector_ip = 10.0.0.129:8086 collector_list= [ [ (*_iter40) = 10.0.0.129:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ConfigCassInitErrorMessage [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 4 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigClientMgrDebug [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 398 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigClientMgrWarning [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 5 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigurationMalformedReferenceWarningLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 4 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = DnsOperational [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 14 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapLinkOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 149 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapNodeOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 211 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 14 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 12 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 8 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveDnsInfo [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 795, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 27, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-06-02 Mon 02:08:04:545.270 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241298810624, Pid 1126454]: Wrong state: Connect for event: EvSandeshSend message: SandeshModuleClientTrace 2025-06-02 Mon 02:08:04:589.231 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241294612224, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:08:04:589.378 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241294612224, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating feature-config:default-global-system-config:ar-client-spine:assisted-replicator controller/src/ifmap/ifmap_server_table.cc 95 2025-06-02 Mon 02:08:04:589.414 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241294612224, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature-config controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:08:04:722.076 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241349191424, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:08:04:722.242 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241349191424, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating feature-config:default-global-system-config:ar-client-leaf:assisted-replicator controller/src/ifmap/ifmap_server_table.cc 95 2025-06-02 Mon 02:08:04:722.293 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241349191424, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature-config controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:08:04:850.111 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241336596224, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:08:04:850.392 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241336596224, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating node-profile:default-global-system-config:juniper-qfx5k-lean controller/src/ifmap/ifmap_server_table.cc 95 2025-06-02 Mon 02:08:04:850.451 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241336596224, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-node-profile controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:08:05:035.400 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241332397824, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:08:05:040.250 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241332397824, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating node-profile:default-global-system-config:juniper-qfx5k controller/src/ifmap/ifmap_server_table.cc 95 2025-06-02 Mon 02:08:05:040.306 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241332397824, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-node-profile controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:08:05:167.405 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241290413824, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:08:05:167.575 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241290413824, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating node-profile:default-global-system-config:juniper-qfx10k-lean controller/src/ifmap/ifmap_server_table.cc 95 2025-06-02 Mon 02:08:05:167.631 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241290413824, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-node-profile controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:08:05:291.386 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241344993024, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:08:05:291.758 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241344993024, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating node-profile:default-global-system-config:juniper-qfx5120 controller/src/ifmap/ifmap_server_table.cc 95 2025-06-02 Mon 02:08:05:291.811 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241344993024, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-node-profile controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:08:05:424.360 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241294612224, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:08:05:424.571 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241294612224, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating node-profile:default-global-system-config:juniper-qfx5220 controller/src/ifmap/ifmap_server_table.cc 95 2025-06-02 Mon 02:08:05:424.625 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241294612224, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-node-profile controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:08:05:527.540 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241303009024, Pid 1126454]: Processing scm::EvSandeshSend in state Connect 2025-06-02 Mon 02:08:05:527.704 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241303009024, Pid 1126454]: SANDESH: Send FAILED: 1748830085527392 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-juju-os-542-1:Control:contrail-dns:0 sm_queue_count = 0 sm_stats= [ ev_stats= [ [ [ event = scm::EvCollectorUpdate enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvConnectTimerExpired enqueues = 4 dequeues = 4 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvIdleHoldTimerExpired enqueues = 4 dequeues = 4 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvSandeshSend enqueues = 29 dequeues = 29 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvStart enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpDeleteSession enqueues = 4 dequeues = 4 enqueue_fails = 0 dequeue_fails = 0 ], [ event = enqueues = 43 dequeues = 43 enqueue_fails = 0 dequeue_fails = 0 ], ] ] state = Connect state_since = 1748830084542669 last_state = Connect last_event = scm::EvSandeshSend last_event_at = 1748830084544738 ] session_stats= [ num_recv_msg = 0 num_recv_msg_fail = 0 num_recv_fail = 0 num_send_msg = 0 num_send_msg_fail = 0 num_send_buffer_fail = 0 num_wait_msgq_enqueue = 0 num_wait_msgq_dequeue = 0 num_write_ready_cb_error = 0 ] session_rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] session_tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] max_sm_queue_count = 5 ] 2025-06-02 Mon 02:08:05:527.737 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241303009024, Pid 1126454]: Wrong state: Connect for event: EvSandeshSend message: SandeshModuleClientTrace 2025-06-02 Mon 02:08:05:527.752 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241303009024, Pid 1126454]: Processing scm::EvSandeshSend in state Connect 2025-06-02 Mon 02:08:05:528.060 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241303009024, Pid 1126454]: SANDESH: Send FAILED: 1748830085527505 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-juju-os-542-1:Control:contrail-dns:0 client_info= [ status = Connect successful_connections = 0 pid = 1126454 http_port = 8092 start_time = 1748830000967073 collector_name = collector_ip = 10.0.0.129:8086 collector_list= [ [ (*_iter40) = 10.0.0.129:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ConfigCassInitErrorMessage [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 4 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigClientMgrDebug [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 405 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigClientMgrWarning [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 5 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigurationMalformedReferenceWarningLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 4 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = DnsOperational [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 14 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapLinkOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 156 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapNodeOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 218 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 15 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 13 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 8 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveDnsInfo [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 816, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 29, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-06-02 Mon 02:08:05:528.091 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241303009024, Pid 1126454]: Wrong state: Connect for event: EvSandeshSend message: SandeshModuleClientTrace 2025-06-02 Mon 02:08:05:566.276 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241294612224, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:08:05:566.447 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241294612224, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating node-profile:default-global-system-config:juniper-qfx10k controller/src/ifmap/ifmap_server_table.cc 95 2025-06-02 Mon 02:08:05:566.586 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241294612224, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-node-profile controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:08:05:768.472 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140240543016704, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:08:05:768.686 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140240543016704, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating node-profile:default-global-system-config:juniper-mx controller/src/ifmap/ifmap_server_table.cc 95 2025-06-02 Mon 02:08:05:768.734 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140240543016704, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-node-profile controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:08:05:907.912 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241332397824, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:08:05:908.168 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241332397824, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating node-profile:default-global-system-config:juniper-srx controller/src/ifmap/ifmap_server_table.cc 95 2025-06-02 Mon 02:08:05:908.226 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241332397824, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-node-profile controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:08:06:037.385 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241340794624, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:08:06:037.546 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241340794624, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating node-profile:default-global-system-config:device-functional-group controller/src/ifmap/ifmap_server_table.cc 95 2025-06-02 Mon 02:08:06:037.582 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241340794624, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-node-profile controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:08:06:225.081 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241319802624, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:08:06:225.346 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241332397824, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating telemetry-profile:default-domain:default-project:default-telemetry-profile-1 controller/src/ifmap/ifmap_server_table.cc 95 2025-06-02 Mon 02:08:06:225.408 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241332397824, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating project-telemetry-profile controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:08:06:345.318 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241311405824, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:08:06:345.500 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241311405824, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating telemetry-profile:default-domain:default-project:default-telemetry-profile-2 controller/src/ifmap/ifmap_server_table.cc 95 2025-06-02 Mon 02:08:06:345.535 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241311405824, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating project-telemetry-profile controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:08:06:481.104 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241303009024, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:08:06:481.260 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241298810624, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating telemetry-profile:default-domain:default-project:default-telemetry-profile-3 controller/src/ifmap/ifmap_server_table.cc 95 2025-06-02 Mon 02:08:06:481.318 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241298810624, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating project-telemetry-profile controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:08:06:677.448 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241290413824, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:08:06:677.908 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241290413824, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating telemetry-profile:default-domain:default-project:default-telemetry-profile-4 controller/src/ifmap/ifmap_server_table.cc 95 2025-06-02 Mon 02:08:06:677.953 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241290413824, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating project-telemetry-profile controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:08:06:806.368 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241340794624, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:08:06:806.548 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241340794624, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating telemetry-profile:default-domain:default-project:default-telemetry-profile-5 controller/src/ifmap/ifmap_server_table.cc 95 2025-06-02 Mon 02:08:06:806.630 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241340794624, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating project-telemetry-profile controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:08:06:956.139 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140240543016704, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:08:06:956.316 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140240543016704, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating telemetry-profile:default-domain:default-project:default-telemetry-profile-6 controller/src/ifmap/ifmap_server_table.cc 95 2025-06-02 Mon 02:08:06:956.363 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140240543016704, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating project-telemetry-profile controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:08:07:093.007 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241294612224, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:08:07:093.151 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241294612224, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating sflow-profile:default-domain:default-project:sflow-all-interfaces controller/src/ifmap/ifmap_server_table.cc 95 2025-06-02 Mon 02:08:07:093.196 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241294612224, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating project-sflow-profile controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:08:07:235.550 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241307207424, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:08:07:235.751 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241349191424, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating sflow-profile:default-domain:default-project:sflow-fabric-interfaces controller/src/ifmap/ifmap_server_table.cc 95 2025-06-02 Mon 02:08:07:235.817 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241349191424, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating project-sflow-profile controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:08:07:361.761 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241336596224, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:08:07:361.942 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241336596224, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating sflow-profile:default-domain:default-project:sflow-access-interfaces controller/src/ifmap/ifmap_server_table.cc 95 2025-06-02 Mon 02:08:07:361.986 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241336596224, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating project-sflow-profile controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:08:07:614.235 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241332397824, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:08:07:614.452 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241332397824, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating grpc-profile:default-domain:default-project:grpc-default-profile controller/src/ifmap/ifmap_server_table.cc 95 2025-06-02 Mon 02:08:07:614.489 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241332397824, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating project-grpc-profile controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:08:07:760.539 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241328199424, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:08:07:762.186 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241328199424, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating snmp-profile:default-domain:default-project:snmp-default-profile controller/src/ifmap/ifmap_server_table.cc 95 2025-06-02 Mon 02:08:07:762.229 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241328199424, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating project-snmp-profile controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:08:07:884.277 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241340794624, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:08:07:884.499 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241340794624, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating netconf-profile:default-domain:default-project:netconf-default-profile controller/src/ifmap/ifmap_server_table.cc 95 2025-06-02 Mon 02:08:07:884.549 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241340794624, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating project-netconf-profile controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:08:08:001.345 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241303009024, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:08:08:004.385 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241303009024, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating device-functional-group:default-domain:default-project:L2-Server-Leaf controller/src/ifmap/ifmap_server_table.cc 95 2025-06-02 Mon 02:08:08:004.452 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241303009024, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating project-device-functional-group controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:08:08:133.642 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241298810624, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:08:08:133.823 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241298810624, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating device-functional-group:default-domain:default-project:L3-Server-Leaf controller/src/ifmap/ifmap_server_table.cc 95 2025-06-02 Mon 02:08:08:133.861 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241298810624, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating project-device-functional-group controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:08:08:263.087 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241344993024, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:08:08:263.296 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241344993024, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating device-functional-group:default-domain:default-project:L3-Gateway-Spine controller/src/ifmap/ifmap_server_table.cc 95 2025-06-02 Mon 02:08:08:263.333 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241344993024, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating project-device-functional-group controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:08:08:385.152 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241315604224, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:08:08:385.348 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241315604224, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating device-functional-group:default-domain:default-project:L3-Storage-Leaf controller/src/ifmap/ifmap_server_table.cc 95 2025-06-02 Mon 02:08:08:385.395 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241315604224, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating project-device-functional-group controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:08:08:511.527 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241324001024, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:08:08:511.712 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241324001024, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating device-functional-group:default-domain:default-project:L3-Server-Leaf-with-Optimized-Multicast controller/src/ifmap/ifmap_server_table.cc 95 2025-06-02 Mon 02:08:08:511.753 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241324001024, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating project-device-functional-group controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:08:08:647.566 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241344993024, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:08:08:647.952 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241328199424, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating device-functional-group:default-domain:default-project:Centrally-Routed-Border-Spine controller/src/ifmap/ifmap_server_table.cc 95 2025-06-02 Mon 02:08:08:648.006 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241328199424, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating project-device-functional-group controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:08:08:785.883 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241332397824, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:08:08:786.086 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241332397824, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating device-functional-group:default-domain:default-project:Centrally-Routed-Border-Spine-With-Optimized-Multicast controller/src/ifmap/ifmap_server_table.cc 95 2025-06-02 Mon 02:08:08:786.130 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241332397824, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating project-device-functional-group controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:08:08:928.439 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241311405824, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:08:08:928.628 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241311405824, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating device-functional-group:default-domain:default-project:Border-Spine-in-Edge-Routed controller/src/ifmap/ifmap_server_table.cc 95 2025-06-02 Mon 02:08:08:928.665 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241311405824, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating project-device-functional-group controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:08:09:046.531 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241303009024, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:08:09:046.716 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241303009024, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating device-functional-group:default-domain:default-project:Border-Leaf-in-Edge-Routed controller/src/ifmap/ifmap_server_table.cc 95 2025-06-02 Mon 02:08:09:046.765 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241303009024, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating project-device-functional-group controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:08:09:176.106 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241315604224, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:08:09:177.153 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241315604224, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating device-functional-group:default-domain:default-project:Lean-Spine-with-Route-Reflector controller/src/ifmap/ifmap_server_table.cc 95 2025-06-02 Mon 02:08:09:177.317 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241315604224, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating project-device-functional-group controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:08:09:306.802 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241349191424, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:08:09:306.953 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241349191424, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating role-config:default-global-system-config:juniper-qfx5k-lean:basic controller/src/ifmap/ifmap_server_table.cc 95 2025-06-02 Mon 02:08:09:306.988 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241349191424, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating node-profile-role-config controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:08:09:347.221 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241507256448, Pid 1126454]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.0.0.129:8092::10.0.0.129:52220 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 256 2025-06-02 Mon 02:08:09:401.642 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241507256448, Pid 1126454]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.0.0.129:8092::10.0.0.129:52220 < Read failed due to error asio.ssl 335544539 : short read src/contrail-common/io/tcp_session.cc 504 2025-06-02 Mon 02:08:09:429.002 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241294612224, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:08:09:429.188 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241332397824, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating role-config:default-global-system-config:juniper-qfx5k:basic controller/src/ifmap/ifmap_server_table.cc 95 2025-06-02 Mon 02:08:09:429.270 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241332397824, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating node-profile-role-config controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:08:09:561.891 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241324001024, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:08:09:562.059 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241324001024, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating role-config:default-global-system-config:juniper-qfx5120:basic controller/src/ifmap/ifmap_server_table.cc 95 2025-06-02 Mon 02:08:09:562.199 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241324001024, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating node-profile-role-config controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:08:09:688.616 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241324001024, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:08:09:688.817 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241324001024, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating role-config:default-global-system-config:juniper-qfx5220:basic controller/src/ifmap/ifmap_server_table.cc 95 2025-06-02 Mon 02:08:09:688.863 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241324001024, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating node-profile-role-config controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:08:09:811.318 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241324001024, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:08:09:811.499 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241324001024, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating role-config:default-global-system-config:juniper-qfx10k-lean:basic controller/src/ifmap/ifmap_server_table.cc 95 2025-06-02 Mon 02:08:09:811.534 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241324001024, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating node-profile-role-config controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:08:09:936.023 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241303009024, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:08:09:936.226 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241294612224, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating role-config:default-global-system-config:juniper-qfx10k:basic controller/src/ifmap/ifmap_server_table.cc 95 2025-06-02 Mon 02:08:09:936.281 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241294612224, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating node-profile-role-config controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:08:10:060.300 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241332397824, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:08:10:060.450 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241332397824, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating role-config:default-global-system-config:juniper-mx:basic controller/src/ifmap/ifmap_server_table.cc 95 2025-06-02 Mon 02:08:10:060.492 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241332397824, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating node-profile-role-config controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:08:10:186.161 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241319802624, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:08:10:186.321 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241319802624, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating role-config:default-global-system-config:juniper-srx:basic controller/src/ifmap/ifmap_server_table.cc 95 2025-06-02 Mon 02:08:10:186.821 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241319802624, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating node-profile-role-config controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:08:10:352.490 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241319802624, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:08:10:352.700 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241319802624, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:08:10:513.098 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241336596224, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:08:10:513.260 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241336596224, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:08:10:673.269 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241336596224, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:08:10:674.052 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241336596224, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:08:10:793.024 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241507256448, Pid 1126454]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.0.0.129:8092::10.0.0.129:52228 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 256 2025-06-02 Mon 02:08:10:834.830 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241307207424, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:08:10:835.009 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241307207424, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:08:10:864.171 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241507256448, Pid 1126454]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.0.0.129:8092::10.0.0.129:52228 < Read failed due to error asio.ssl 335544539 : short read src/contrail-common/io/tcp_session.cc 504 2025-06-02 Mon 02:08:11:010.309 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140240543016704, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:08:11:010.834 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241319802624, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:08:11:163.573 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241319802624, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:08:11:163.784 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241319802624, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:08:11:334.248 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241294612224, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:08:11:334.454 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241344993024, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:08:11:487.379 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241332397824, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:08:11:488.178 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241332397824, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:08:11:646.847 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140240543016704, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:08:11:647.068 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241328199424, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-physical-role controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:08:11:812.489 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241324001024, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:08:11:813.079 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241324001024, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-overlay-role controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:08:11:970.868 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241324001024, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:08:11:971.158 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241290413824, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:08:12:123.830 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241349191424, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:08:12:124.006 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241349191424, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:08:12:277.278 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241332397824, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:08:12:277.452 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241332397824, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:08:12:434.384 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241298810624, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:08:12:434.555 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241298810624, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:08:12:594.243 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241336596224, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:08:12:594.413 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241336596224, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:08:12:746.822 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241340794624, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:08:12:747.203 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241340794624, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:08:12:914.649 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241290413824, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:08:12:915.656 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140240543016704, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-physical-role controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:08:13:064.261 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241349191424, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:08:13:064.441 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241349191424, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-overlay-role controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:08:13:225.940 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241294612224, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:08:13:226.141 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241332397824, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:08:13:381.075 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140240543016704, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:08:13:381.245 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140240543016704, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:08:13:536.649 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241328199424, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:08:13:536.832 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241328199424, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:08:13:694.584 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241311405824, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:08:13:694.831 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241319802624, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:08:13:850.223 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241340794624, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:08:13:850.403 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241340794624, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:08:14:013.978 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241294612224, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:08:14:014.161 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241294612224, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:08:14:177.072 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241336596224, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:08:14:177.261 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241336596224, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-physical-role controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:08:14:333.969 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241349191424, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:08:14:334.143 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241349191424, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-overlay-role controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:08:14:492.703 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241303009024, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:08:14:492.928 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241303009024, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:08:14:653.385 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241340794624, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:08:14:653.549 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241340794624, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:08:14:820.548 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241344993024, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:08:14:820.745 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241344993024, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:08:14:979.126 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241340794624, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:08:14:979.344 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241332397824, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:08:15:139.977 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241336596224, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:08:15:140.173 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241336596224, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:08:15:285.600 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241290413824, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:08:15:285.780 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241290413824, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:08:15:453.221 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241319802624, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:08:15:453.421 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241319802624, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-overlay-role controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:08:15:610.318 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241319802624, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:08:15:610.530 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241349191424, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-physical-role controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:08:15:822.575 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241307207424, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:08:15:822.786 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241307207424, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:08:15:998.217 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241298810624, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:08:15:998.475 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241298810624, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:08:16:164.760 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241340794624, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:08:16:164.972 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241336596224, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:08:16:336.448 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241311405824, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:08:16:336.666 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241311405824, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:08:16:491.162 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241298810624, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:08:16:491.331 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241298810624, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:08:16:651.083 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241303009024, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:08:16:651.286 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241303009024, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:08:16:816.672 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241298810624, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:08:16:816.870 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241298810624, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-overlay-role controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:08:16:977.308 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241324001024, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:08:16:977.495 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241324001024, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-physical-role controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:08:17:134.854 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241349191424, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:08:17:135.027 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241298810624, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:08:17:288.042 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241344993024, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:08:17:288.257 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140240543016704, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:08:17:439.261 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241336596224, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:08:17:439.429 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241336596224, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:08:17:591.824 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241349191424, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:08:17:592.963 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241315604224, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:08:17:747.802 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241298810624, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:08:17:747.960 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241298810624, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:08:17:914.143 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241328199424, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:08:17:915.571 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241332397824, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-overlay-role controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:08:18:087.617 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140240543016704, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:08:18:087.844 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140240543016704, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-physical-role controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:08:18:252.256 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241328199424, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:08:18:252.476 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140240543016704, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:08:18:418.999 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140240543016704, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:08:18:419.229 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140240543016704, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:08:18:578.230 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241298810624, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:08:18:578.469 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241298810624, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:08:18:744.564 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241315604224, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:08:18:744.818 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241315604224, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:08:18:906.882 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241311405824, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:08:18:907.078 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241294612224, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:08:19:063.099 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241294612224, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:08:19:063.275 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241294612224, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-overlay-role controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:08:19:231.105 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241332397824, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:08:19:231.781 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241332397824, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-physical-role controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:08:19:395.619 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241328199424, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:08:19:395.783 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241328199424, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:08:19:554.896 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241294612224, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:08:19:555.082 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241294612224, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:08:19:710.470 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140240543016704, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:08:19:710.625 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140240543016704, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:08:19:878.217 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241344993024, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:08:19:878.416 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241344993024, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:08:20:049.612 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241319802624, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:08:20:049.804 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241319802624, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:08:20:216.075 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140240543016704, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:08:20:216.253 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140240543016704, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:08:20:369.461 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241298810624, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:08:20:369.760 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241298810624, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:08:20:530.377 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241298810624, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:08:20:530.547 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241298810624, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-overlay-role controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:08:20:547.399 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241328199424, Pid 1126454]: 10.0.0.129:8086 EvConnectTimerExpired in state Connect 2025-06-02 Mon 02:08:20:547.447 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241328199424, Pid 1126454]: Processing scm::EvConnectTimerExpired in state Connect 2025-06-02 Mon 02:08:20:547.456 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241328199424, Pid 1126454]: Connect : EvConnectTimerExpired 2025-06-02 Mon 02:08:20:547.580 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241328199424, Pid 1126454]: Idle 2025-06-02 Mon 02:08:20:547.611 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241507256448, Pid 1126454]: TCP [SYS_ERR]: TcpSessionMessageLog: Session -(-1) > SSL Handshake failed due to error: 125 category: system message: Operation canceled src/contrail-common/io/ssl_server.cc 134 2025-06-02 Mon 02:08:20:547.775 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241328199424, Pid 1126454]: Processing scm::EvSandeshSend in state Idle 2025-06-02 Mon 02:08:20:547.807 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241328199424, Pid 1126454]: SANDESH: Send FAILED: 1748830100547511 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-juju-os-542-1 process_status= [ [ [ module_id = contrail-dns instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvConnectTimerExpired ], [ type = Database name = Cassandra server_addrs= [ [ (*_iter6) = 10.0.0.129:9041, ] ] status = Up description = Established Cassandra connection ], [ type = Database name = RabbitMQ server_addrs= [ [ (*_iter6) = 10.0.0.129:5673, ] ] status = Up description = RabbitMQ connection established ], ] ] flag_infos= [ [ ] ] description = Collector connection down ], ] ] ] 2025-06-02 Mon 02:08:20:547.824 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241328199424, Pid 1126454]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-06-02 Mon 02:08:20:547.833 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241328199424, Pid 1126454]: Processing scm::EvTcpDeleteSession in state Idle 2025-06-02 Mon 02:08:20:547.867 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241328199424, Pid 1126454]: Processing scm::EvSandeshSend in state Idle 2025-06-02 Mon 02:08:20:548.065 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241328199424, Pid 1126454]: SANDESH: Send FAILED: 1748830100547758 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-juju-os-542-1:Control:contrail-dns:0 client_info= [ status = Idle successful_connections = 0 pid = 1126454 http_port = 8092 start_time = 1748830000967073 collector_name = collector_ip = 10.0.0.129:8086 collector_list= [ [ (*_iter40) = 10.0.0.129:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ConfigCassInitErrorMessage [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 4 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigClientMgrDebug [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 503 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigClientMgrWarning [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 5 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigurationMalformedReferenceWarningLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 4 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = DnsOperational [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 14 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapLinkOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 254 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapNodeOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 252 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 15 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 15 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 13 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveDnsInfo [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 1051, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 31, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-06-02 Mon 02:08:20:548.086 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241328199424, Pid 1126454]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-06-02 Mon 02:08:20:697.166 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241303009024, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:08:20:697.321 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241303009024, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-physical-role controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:08:20:863.793 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241328199424, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:08:20:869.374 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241328199424, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-physical-role controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:08:21:041.435 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241340794624, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:08:21:041.589 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241340794624, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-overlay-role controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:08:21:226.245 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241344993024, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:08:21:226.529 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241344993024, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:08:21:382.192 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241324001024, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:08:21:382.435 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241298810624, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:08:21:553.009 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241290413824, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:08:21:553.423 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241319802624, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:08:21:722.915 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241349191424, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:08:21:723.081 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241349191424, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-overlay-role controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:08:21:885.158 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241290413824, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:08:21:885.324 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241290413824, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-physical-role controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:08:22:045.844 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241319802624, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:08:22:046.077 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241319802624, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:08:22:205.507 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241294612224, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:08:22:205.759 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241298810624, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:08:22:361.577 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241344993024, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:08:22:366.190 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241344993024, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:08:22:516.327 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241336596224, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:08:22:516.490 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241336596224, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:08:22:667.552 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140240543016704, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:08:22:667.765 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140240543016704, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:08:22:837.842 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241298810624, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:08:22:838.655 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241319802624, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:08:23:008.768 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241315604224, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:08:23:008.961 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241315604224, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:08:23:172.713 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241319802624, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:08:23:176.348 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241319802624, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:08:23:546.922 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241307207424, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:08:23:547.094 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241307207424, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-overlay-role controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:08:23:706.219 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241340794624, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:08:23:706.970 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241340794624, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-physical-role controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:08:23:860.465 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241319802624, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:08:23:860.636 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241319802624, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:08:24:018.174 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241315604224, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:08:24:018.378 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241336596224, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:08:24:184.724 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241298810624, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:08:24:184.965 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241303009024, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:08:24:345.918 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241324001024, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:08:24:346.141 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241324001024, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:08:24:521.465 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241303009024, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:08:24:521.673 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241303009024, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:08:24:682.326 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241290413824, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:08:24:682.504 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241290413824, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:08:24:698.628 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241507256448, Pid 1126454]: XMPP [SYS_DEBUG]: XmppCreateConnection: XMPP Peer cn-jenkins-deploy-platform-juju-os-542-1:10.0.0.129 SEND Xmpp creating dynamic channel 10.0.0.129:8093::10.0.0.129:37735 controller/src/xmpp/xmpp_server.cc 578 2025-06-02 Mon 02:08:24:700.027 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241507256448, Pid 1126454]: XMPP [SYS_INFO]: XmppConnectionCreate: XMPP Peer cn-jenkins-deploy-platform-juju-os-542-1:10.0.0.129 RECV Created Xmpp Server connection from cn-jenkins-deploy-platform-juju-os-542-1 To : controller/src/xmpp/xmpp_connection.cc 727 2025-06-02 Mon 02:08:24:701.058 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241507256448, Pid 1126454]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.0.0.129:8093::10.0.0.129:37735 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 256 2025-06-02 Mon 02:08:24:701.096 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241328199424, Pid 1126454]: Processing scm::EvSandeshSend in state Idle 2025-06-02 Mon 02:08:24:701.117 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241340794624, Pid 1126454]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer cn-jenkins-deploy-platform-juju-os-542-1:10.0.0.129 RECV Mode Server: PassiveOpen in state: Idle peer ip: 10.0.0.129 ( ) controller/src/xmpp/xmpp_state_machine.cc 1325 2025-06-02 Mon 02:08:24:701.866 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241328199424, Pid 1126454]: SANDESH: Send FAILED: 1748830104701070 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.0.0.129 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-06-02 Mon 02:08:24:701.887 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241328199424, Pid 1126454]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-06-02 Mon 02:08:24:702.910 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241328199424, Pid 1126454]: Processing scm::EvSandeshSend in state Idle 2025-06-02 Mon 02:08:24:702.943 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241328199424, Pid 1126454]: SANDESH: Send FAILED: 1748830104702356 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.0.0.129 state = Active event = EvTcpPassiveOpen local_port = 8093 remote_port = 37735 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-06-02 Mon 02:08:24:702.952 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241328199424, Pid 1126454]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-06-02 Mon 02:08:24:703.461 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241349191424, Pid 1126454]: Processing scm::EvSandeshSend in state Idle 2025-06-02 Mon 02:08:24:703.497 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241349191424, Pid 1126454]: SANDESH: Send FAILED: 1748830104703441 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.0.0.129 state = Active next_state = Open Confirm event = EvXmppOpen identifier = cn-jenkins-deploy-platform-juju-os-542-1/dns ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-06-02 Mon 02:08:24:703.506 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241349191424, Pid 1126454]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-06-02 Mon 02:08:24:705.529 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241315604224, Pid 1126454]: XMPP [SYS_INFO]: XmppUnconsumedEvent: XMPP Peer cn-jenkins-deploy-platform-juju-os-542-1:10.0.0.129 RECV Mode Server: Unconsumed Event xmsm::EvXmppKeepalive in state OpenConfirm controller/src/xmpp/xmpp_state_machine.cc 1649 2025-06-02 Mon 02:08:24:705.560 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241315604224, Pid 1126454]: Processing scm::EvSandeshSend in state Idle 2025-06-02 Mon 02:08:24:705.577 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241315604224, Pid 1126454]: SANDESH: Send FAILED: 1748830104705493 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.0.0.129 state = OpenConfirm next_state = Trigger Server Ssl Handshake event = EvStartTls identifier = cn-jenkins-deploy-platform-juju-os-542-1/dns ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-06-02 Mon 02:08:24:705.584 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241315604224, Pid 1126454]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-06-02 Mon 02:08:24:710.648 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241507256448, Pid 1126454]: XMPP [SYS_DEBUG]: XmppSslHandShakeMessage: XMPP Peer cn-jenkins-deploy-platform-juju-os-542-1:10.0.0.129 RECV success controller/src/xmpp/xmpp_connection.cc 402 2025-06-02 Mon 02:08:24:710.887 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241307207424, Pid 1126454]: Processing scm::EvSandeshSend in state Idle 2025-06-02 Mon 02:08:24:710.942 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241307207424, Pid 1126454]: SANDESH: Send FAILED: 1748830104710858 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.0.0.129 state = OpenConfirm next_state = OpenConfirm Feature Negotiation Success event = EvTlsHandShakeSuccess ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-06-02 Mon 02:08:24:711.021 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241307207424, Pid 1126454]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-06-02 Mon 02:08:24:711.600 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241324001024, Pid 1126454]: Processing scm::EvSandeshSend in state Idle 2025-06-02 Mon 02:08:24:711.640 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241324001024, Pid 1126454]: SANDESH: Send FAILED: 1748830104711567 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.0.0.129 state = OpenConfirm next_state = Established event = EvXmppOpen identifier = cn-jenkins-deploy-platform-juju-os-542-1/dns ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-06-02 Mon 02:08:24:711.662 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241324001024, Pid 1126454]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-06-02 Mon 02:08:24:839.449 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241324001024, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:08:24:839.692 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241344993024, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:08:25:001.893 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241298810624, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:08:25:002.075 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241298810624, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-overlay-role controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:08:25:156.169 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241340794624, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:08:25:156.337 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241340794624, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-physical-role controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:08:25:326.376 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241336596224, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:08:25:326.572 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241336596224, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:08:25:486.442 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241319802624, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:08:25:486.666 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241344993024, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:08:25:648.070 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241328199424, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:08:25:648.354 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241324001024, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:08:25:810.361 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241290413824, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:08:25:810.588 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241349191424, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:08:25:971.194 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241290413824, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:08:25:971.464 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140240543016704, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:08:26:129.245 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241340794624, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:08:26:129.444 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241340794624, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:08:26:282.748 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241315604224, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:08:26:282.925 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241315604224, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:08:26:487.475 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241344993024, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:08:26:487.724 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241344993024, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-overlay-role controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:08:26:622.579 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241328199424, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:08:26:622.940 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241328199424, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-physical-role controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:08:26:779.689 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241315604224, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:08:26:937.790 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241328199424, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:08:26:937.961 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241328199424, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:08:27:098.907 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241307207424, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:08:27:099.147 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241307207424, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:08:27:265.549 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241332397824, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:08:27:265.750 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241332397824, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:08:27:426.794 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241303009024, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:08:27:427.005 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241303009024, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:08:27:576.237 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241324001024, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:08:27:576.426 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241324001024, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:08:27:745.450 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241336596224, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:08:27:745.634 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241336596224, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:08:27:911.651 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241349191424, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:08:27:911.810 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241349191424, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:08:28:062.185 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241324001024, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:08:28:062.373 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241324001024, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:08:28:233.869 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241319802624, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:08:28:234.983 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241319802624, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:08:28:389.045 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241303009024, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:08:28:389.244 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241303009024, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:08:28:548.586 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241332397824, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:08:28:548.848 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241324001024, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-physical-role controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:08:28:705.656 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241303009024, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:08:28:705.832 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241303009024, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-overlay-role controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:08:28:750.346 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241307207424, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:08:28:860.561 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241290413824, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:08:28:860.907 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241307207424, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-physical-role controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:08:28:875.525 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241349191424, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:08:28:875.682 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241349191424, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating virtual-router:default-global-system-config:cn-jenkins-deploy-platform-juju-os-542-1 controller/src/ifmap/ifmap_server_table.cc 95 2025-06-02 Mon 02:08:28:875.720 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241349191424, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating global-system-config-virtual-router controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:08:29:027.523 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241311405824, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:08:29:027.702 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241311405824, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-overlay-role controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:08:29:187.439 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241349191424, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:08:29:187.622 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241349191424, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-physical-role controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:08:29:211.504 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241290413824, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:08:29:211.724 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241294612224, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating virtual-machine-interface:default-global-system-config:cn-jenkins-deploy-platform-juju-os-542-1:vhost0 controller/src/ifmap/ifmap_server_table.cc 95 2025-06-02 Mon 02:08:29:211.813 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241294612224, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating virtual-router-virtual-machine-interface controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:08:29:211.870 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241294612224, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating virtual-machine-interface-virtual-network controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:08:29:244.835 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241340794624, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:08:29:257.918 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241332397824, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:08:29:357.476 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241336596224, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:08:29:357.714 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241336596224, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-overlay-role controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:08:29:510.443 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241340794624, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:08:29:510.657 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241340794624, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-physical-role controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:08:29:701.407 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140240543016704, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:08:29:701.576 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140240543016704, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-physical-role controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:08:29:849.647 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241324001024, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:08:29:849.845 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140240543016704, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-physical-role controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:08:30:014.669 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241349191424, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:08:30:014.842 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241349191424, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-physical-role controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:08:30:171.722 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241303009024, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:08:30:171.919 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241303009024, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-overlay-role controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:08:30:326.304 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241324001024, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:08:30:326.470 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241324001024, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-overlay-role controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:08:30:477.854 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241336596224, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:08:30:478.030 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241336596224, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:08:30:640.409 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241298810624, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:08:30:640.621 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241298810624, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:08:30:801.030 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241332397824, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:08:30:801.242 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241332397824, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:08:30:959.431 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241349191424, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:08:30:959.584 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241349191424, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:08:31:119.348 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241336596224, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:08:31:119.559 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241336596224, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-overlay-role controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:08:31:324.091 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241344993024, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:08:31:324.300 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241344993024, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:08:31:492.440 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140240543016704, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:08:31:492.666 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140240543016704, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:08:31:633.822 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241303009024, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:08:31:634.051 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241307207424, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:08:31:786.207 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241315604224, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:08:31:786.411 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241349191424, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:08:31:937.311 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241303009024, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:08:31:937.488 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241303009024, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:08:32:088.384 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241307207424, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:08:32:089.036 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241307207424, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:08:32:244.092 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241319802624, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:08:32:244.296 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241340794624, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-overlay-role controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:08:32:396.905 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241319802624, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:08:32:397.148 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241315604224, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:08:32:554.642 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241303009024, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:08:32:554.865 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241303009024, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:08:32:715.806 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241328199424, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:08:32:716.041 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241344993024, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:08:32:879.176 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241290413824, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:08:32:879.354 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241290413824, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:08:33:034.573 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241340794624, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:08:33:034.796 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241340794624, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:08:33:190.089 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241298810624, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:08:33:190.329 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241319802624, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:08:33:343.675 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140240543016704, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:08:33:343.967 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140240543016704, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:08:33:380.627 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241507256448, Pid 1126454]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.0.0.129:8093::10.0.0.129:37735 < Read failed due to error asio.ssl 335544539 : short read src/contrail-common/io/tcp_session.cc 504 2025-06-02 Mon 02:08:33:380.731 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241507256448, Pid 1126454]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer cn-jenkins-deploy-platform-juju-os-542-1:10.0.0.129 RECV Mode Server: Event: Tcp Connection Closed peer ip: 10.0.0.129 ( cn-jenkins-deploy-platform-juju-os-542-1/dns ) controller/src/xmpp/xmpp_state_machine.cc 1310 2025-06-02 Mon 02:08:33:380.891 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241311405824, Pid 1126454]: XMPP [SYS_INFO]: XmppConnectionDelete: XMPP Peer cn-jenkins-deploy-platform-juju-os-542-1:10.0.0.129 Deleted Xmpp Server connection from cn-jenkins-deploy-platform-juju-os-542-1 To : cn-jenkins-deploy-platform-juju-os-542-1/dns controller/src/xmpp/xmpp_connection.cc 734 2025-06-02 Mon 02:08:33:380.918 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241311405824, Pid 1126454]: Processing scm::EvSandeshSend in state Idle 2025-06-02 Mon 02:08:33:380.935 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241311405824, Pid 1126454]: SANDESH: Send FAILED: 1748830113380815 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.0.0.129 state = Established next_state = Idle event = EvTcpClose ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-06-02 Mon 02:08:33:380.950 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241311405824, Pid 1126454]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-06-02 Mon 02:08:33:380.959 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241311405824, Pid 1126454]: Processing scm::EvSandeshSend in state Idle 2025-06-02 Mon 02:08:33:380.970 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241311405824, Pid 1126454]: SANDESH: Send FAILED: 1748830113380824 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.0.0.129 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-06-02 Mon 02:08:33:380.976 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241311405824, Pid 1126454]: Wrong state: Idle for event: EvSandeshSend message: XmppConnectionLog 2025-06-02 Mon 02:08:33:511.000 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241336596224, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:08:33:511.215 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241344993024, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:08:33:672.343 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241303009024, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:08:33:672.576 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241315604224, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:08:33:828.414 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241332397824, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:08:33:828.661 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241332397824, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:08:33:979.282 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241344993024, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:08:33:979.468 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241344993024, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:08:34:141.021 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241298810624, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:08:34:141.201 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241298810624, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:08:34:296.637 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241344993024, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:08:34:296.907 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241298810624, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:08:34:459.996 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241336596224, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:08:34:460.201 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241336596224, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:08:34:614.406 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241328199424, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:08:34:614.630 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241307207424, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:08:34:770.778 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241315604224, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:08:34:770.996 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241315604224, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:08:34:932.506 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241303009024, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:08:34:932.738 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241303009024, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:08:35:081.738 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241349191424, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:08:35:081.922 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241349191424, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:08:35:243.233 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241303009024, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:08:35:243.444 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241349191424, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:08:35:402.196 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140240543016704, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:08:35:402.417 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241298810624, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:08:35:529.087 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241294612224, Pid 1126454]: Processing scm::EvSandeshSend in state Idle 2025-06-02 Mon 02:08:35:529.198 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241294612224, Pid 1126454]: SANDESH: Send FAILED: 1748830115528906 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-juju-os-542-1:Control:contrail-dns:0 sm_queue_count = 0 sm_stats= [ ev_stats= [ [ [ event = scm::EvCollectorUpdate enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvConnectTimerExpired enqueues = 5 dequeues = 5 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvIdleHoldTimerExpired enqueues = 4 dequeues = 4 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvSandeshSend enqueues = 41 dequeues = 41 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvStart enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpDeleteSession enqueues = 5 dequeues = 5 enqueue_fails = 0 dequeue_fails = 0 ], [ event = enqueues = 57 dequeues = 57 enqueue_fails = 0 dequeue_fails = 0 ], ] ] state = Idle state_since = 1748830100547577 last_state = Idle last_event = scm::EvSandeshSend last_event_at = 1748830113380958 ] max_sm_queue_count = 5 ] 2025-06-02 Mon 02:08:35:529.226 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241294612224, Pid 1126454]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-06-02 Mon 02:08:35:529.237 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241294612224, Pid 1126454]: Processing scm::EvSandeshSend in state Idle 2025-06-02 Mon 02:08:35:529.556 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241294612224, Pid 1126454]: SANDESH: Send FAILED: 1748830115529035 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-juju-os-542-1:Control:contrail-dns:0 client_info= [ status = Idle successful_connections = 0 pid = 1126454 http_port = 8092 start_time = 1748830000967073 collector_name = collector_ip = 10.0.0.129:8086 collector_list= [ [ (*_iter40) = 10.0.0.129:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ConfigCassInitErrorMessage [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 4 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigClientMgrDebug [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 599 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigClientMgrWarning [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 5 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigurationMalformedReferenceWarningLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 4 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = DnsOperational [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 14 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapLinkOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 347 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapNodeOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 254 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 16 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 16 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 15 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveDnsInfo [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionCreate [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionDelete [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 8 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppCreateConnection [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppSslHandShakeMessage [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppUnconsumedEvent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 1251, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 41, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-06-02 Mon 02:08:35:529.581 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241294612224, Pid 1126454]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-06-02 Mon 02:08:35:557.559 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241315604224, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:08:35:557.783 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241315604224, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:08:35:717.463 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241307207424, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:08:35:717.707 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241307207424, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:08:35:871.539 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140240543016704, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:08:35:871.775 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140240543016704, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:08:36:029.163 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241315604224, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:08:36:029.436 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241336596224, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:08:36:181.991 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241344993024, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:08:36:182.177 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241344993024, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:08:36:335.191 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241336596224, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:08:36:335.436 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241336596224, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:08:36:496.646 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241298810624, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:08:36:496.833 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241298810624, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:08:36:547.768 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241315604224, Pid 1126454]: Processing scm::EvIdleHoldTimerExpired in state Idle 2025-06-02 Mon 02:08:36:548.038 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241315604224, Pid 1126454]: Connect : Start Connect timer 10.0.0.129:8086 2025-06-02 Mon 02:08:36:548.173 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241315604224, Pid 1126454]: Processing scm::EvSandeshSend in state Connect 2025-06-02 Mon 02:08:36:548.225 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241315604224, Pid 1126454]: SANDESH: Send FAILED: 1748830116547850 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-juju-os-542-1 process_status= [ [ [ module_id = contrail-dns instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.129:8086, ] ] status = Initializing description = Idle : EvIdleHoldTimerExpired -> Connect ], [ type = Database name = Cassandra server_addrs= [ [ (*_iter6) = 10.0.0.129:9041, ] ] status = Up description = Established Cassandra connection ], [ type = Database name = RabbitMQ server_addrs= [ [ (*_iter6) = 10.0.0.129:5673, ] ] status = Up description = RabbitMQ connection established ], ] ] flag_infos= [ [ ] ] description = Collector connection down ], ] ] ] 2025-06-02 Mon 02:08:36:548.235 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241315604224, Pid 1126454]: Wrong state: Connect for event: EvSandeshSend message: NodeStatusUVE 2025-06-02 Mon 02:08:36:548.243 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241315604224, Pid 1126454]: Processing scm::EvSandeshSend in state Connect 2025-06-02 Mon 02:08:36:548.558 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241315604224, Pid 1126454]: SANDESH: Send FAILED: 1748830116548153 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-juju-os-542-1:Control:contrail-dns:0 client_info= [ status = Connect successful_connections = 0 pid = 1126454 http_port = 8092 start_time = 1748830000967073 collector_name = collector_ip = 10.0.0.129:8086 collector_list= [ [ (*_iter40) = 10.0.0.129:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ConfigCassInitErrorMessage [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 4 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigClientMgrDebug [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 606 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigClientMgrWarning [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 5 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigurationMalformedReferenceWarningLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 4 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = DnsOperational [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 14 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapLinkOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 354 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapNodeOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 254 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 16 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 18 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 15 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveDnsInfo [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionCreate [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionDelete [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 8 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppCreateConnection [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppSslHandShakeMessage [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppUnconsumedEvent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 1265, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 43, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-06-02 Mon 02:08:36:548.587 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241315604224, Pid 1126454]: Wrong state: Connect for event: EvSandeshSend message: SandeshModuleClientTrace 2025-06-02 Mon 02:08:36:655.835 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241315604224, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:08:36:656.004 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241315604224, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:08:36:810.306 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241349191424, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:08:36:810.488 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241328199424, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:08:36:965.708 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241303009024, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:08:36:965.926 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241294612224, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:08:37:120.723 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241303009024, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:08:37:121.071 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241303009024, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:08:37:285.514 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241336596224, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:08:37:285.706 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241349191424, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:08:37:445.944 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241344993024, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:08:37:446.128 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241290413824, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:08:37:598.508 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241344993024, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:08:37:598.733 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241344993024, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:08:37:751.176 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241340794624, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:08:37:751.387 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241328199424, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:08:37:912.270 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241294612224, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:08:37:912.438 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241294612224, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:08:38:067.830 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241344993024, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:08:38:068.054 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241307207424, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-overlay-role controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:08:38:235.556 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241328199424, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:08:38:235.828 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241290413824, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-physical-role controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:08:38:389.660 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241298810624, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:08:38:389.826 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241298810624, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:08:38:552.188 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241349191424, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:08:38:552.371 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241349191424, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:08:38:703.739 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241311405824, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:08:38:703.978 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241332397824, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-feature controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:08:38:859.201 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241290413824, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:08:38:859.404 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241290413824, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-physical-role controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:08:39:014.722 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241311405824, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:08:39:014.895 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241311405824, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-physical-role controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:08:39:165.455 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241336596224, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:08:39:165.641 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241336596224, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-physical-role controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:08:39:335.872 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241324001024, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:08:39:336.072 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241324001024, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-overlay-role controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:08:39:483.793 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241315604224, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:08:39:483.973 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241315604224, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-overlay-role controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:08:39:652.267 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241298810624, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:08:39:652.427 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241298810624, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating role-definition-overlay-role controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:08:39:797.756 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241336596224, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:08:39:797.941 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241336596224, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating node-profile-hardware controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:08:39:958.280 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241344993024, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:08:39:958.531 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241328199424, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating node-profile-hardware controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:08:40:114.704 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241328199424, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:08:40:114.898 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241298810624, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating node-profile-hardware controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:08:40:270.875 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241307207424, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:08:40:271.086 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241307207424, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating node-profile-hardware controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:08:40:429.152 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241294612224, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:08:40:429.317 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241294612224, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating node-profile-hardware controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:08:40:586.361 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241319802624, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:08:40:586.563 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241336596224, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating node-profile-hardware controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:08:40:744.440 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241344993024, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:08:40:744.950 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241344993024, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating node-profile-hardware controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:08:40:904.151 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241344993024, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:08:40:904.327 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241344993024, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating node-profile-hardware controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:08:41:059.725 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241349191424, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:08:41:059.904 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241349191424, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating node-profile-hardware controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:08:41:215.687 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241324001024, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:08:41:215.954 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241324001024, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating node-profile-hardware controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:08:41:385.623 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241294612224, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:08:41:385.798 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241294612224, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating node-profile-hardware controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:08:41:531.072 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241311405824, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:08:41:531.312 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241311405824, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating node-profile-hardware controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:08:41:686.761 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241328199424, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:08:41:686.951 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241328199424, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating node-profile-hardware controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:08:41:838.065 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241336596224, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:08:41:838.361 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241294612224, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating node-profile-hardware controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:08:42:018.143 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241344993024, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:08:42:018.346 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241344993024, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating node-profile-hardware controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:08:42:172.616 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241303009024, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:08:42:172.864 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241303009024, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating node-profile-hardware controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:08:42:334.403 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241324001024, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:08:42:334.634 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140240543016704, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating node-profile-hardware controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:08:42:487.027 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241340794624, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:08:42:487.197 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241340794624, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating node-profile-hardware controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:08:42:661.207 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241332397824, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:08:42:661.443 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241332397824, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating node-profile-hardware controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:08:42:812.392 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241311405824, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:08:42:812.644 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241349191424, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating node-profile-hardware controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:08:42:991.514 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241303009024, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:08:42:991.756 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241303009024, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating node-profile-hardware controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:08:43:147.259 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241315604224, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:08:43:147.477 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241332397824, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating node-profile-hardware controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:08:43:320.623 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241319802624, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:08:43:320.805 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241319802624, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating node-profile-hardware controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:08:43:474.300 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241315604224, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:08:43:480.650 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241315604224, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating node-profile-hardware controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:08:43:653.100 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241349191424, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:08:43:653.300 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241349191424, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating node-profile-hardware controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:08:43:822.922 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241336596224, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:08:43:823.093 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241336596224, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating node-profile-hardware controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:08:43:986.905 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241311405824, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:08:43:987.137 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241311405824, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating node-profile-hardware controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:08:44:155.122 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241336596224, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:08:44:155.303 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241336596224, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating node-profile-hardware controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:08:44:366.894 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241298810624, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:08:44:367.087 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241298810624, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating node-profile-hardware controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:08:44:528.842 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241290413824, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:08:44:529.020 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241290413824, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating node-profile-hardware controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:08:44:694.648 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241349191424, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:08:44:694.790 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241349191424, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating node-profile-hardware controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:08:44:869.613 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241344993024, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:08:44:869.799 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241344993024, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating node-profile-hardware controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:08:45:025.905 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241307207424, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:08:45:026.055 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241307207424, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating node-profile-hardware controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:08:45:193.605 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241311405824, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:08:45:193.788 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241311405824, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating node-profile-hardware controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:08:45:357.417 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241319802624, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:08:45:357.924 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241319802624, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating node-profile-hardware controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:08:45:368.145 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241507256448, Pid 1126454]: XMPP [SYS_DEBUG]: XmppCreateConnection: XMPP Peer cn-jenkins-deploy-platform-juju-os-542-1:10.0.0.129 SEND Xmpp creating dynamic channel 10.0.0.129:8093::10.0.0.129:42709 controller/src/xmpp/xmpp_server.cc 578 2025-06-02 Mon 02:08:45:368.211 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241507256448, Pid 1126454]: XMPP [SYS_INFO]: XmppConnectionCreate: XMPP Peer cn-jenkins-deploy-platform-juju-os-542-1:10.0.0.129 RECV Created Xmpp Server connection from cn-jenkins-deploy-platform-juju-os-542-1 To : controller/src/xmpp/xmpp_connection.cc 727 2025-06-02 Mon 02:08:45:368.248 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241507256448, Pid 1126454]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.0.0.129:8093::10.0.0.129:42709 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 256 2025-06-02 Mon 02:08:45:368.346 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241311405824, Pid 1126454]: XMPP [SYS_NOTICE]: XmppEventLog: XMPP Peer cn-jenkins-deploy-platform-juju-os-542-1:10.0.0.129 RECV Mode Server: PassiveOpen in state: Idle peer ip: 10.0.0.129 ( ) controller/src/xmpp/xmpp_state_machine.cc 1325 2025-06-02 Mon 02:08:45:368.448 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241311405824, Pid 1126454]: Processing scm::EvSandeshSend in state Connect 2025-06-02 Mon 02:08:45:368.490 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241311405824, Pid 1126454]: SANDESH: Send FAILED: 1748830125368305 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.0.0.129 state = Idle next_state = Active event = Start ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-06-02 Mon 02:08:45:368.503 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241311405824, Pid 1126454]: Wrong state: Connect for event: EvSandeshSend message: XmppConnectionLog 2025-06-02 Mon 02:08:45:368.542 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241311405824, Pid 1126454]: Processing scm::EvSandeshSend in state Connect 2025-06-02 Mon 02:08:45:368.557 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241311405824, Pid 1126454]: SANDESH: Send FAILED: 1748830125368435 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.0.0.129 state = Active event = EvTcpPassiveOpen local_port = 8093 remote_port = 42709 ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-06-02 Mon 02:08:45:368.563 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241311405824, Pid 1126454]: Wrong state: Connect for event: EvSandeshSend message: XmppConnectionLog 2025-06-02 Mon 02:08:45:372.441 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241311405824, Pid 1126454]: Processing scm::EvSandeshSend in state Connect 2025-06-02 Mon 02:08:45:372.484 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241311405824, Pid 1126454]: SANDESH: Send FAILED: 1748830125372378 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.0.0.129 state = Active next_state = Open Confirm event = EvXmppOpen identifier = cn-jenkins-deploy-platform-juju-os-542-1/dns ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-06-02 Mon 02:08:45:372.502 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241311405824, Pid 1126454]: Wrong state: Connect for event: EvSandeshSend message: XmppConnectionLog 2025-06-02 Mon 02:08:45:378.817 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241315604224, Pid 1126454]: XMPP [SYS_INFO]: XmppUnconsumedEvent: XMPP Peer cn-jenkins-deploy-platform-juju-os-542-1:10.0.0.129 RECV Mode Server: Unconsumed Event xmsm::EvXmppKeepalive in state OpenConfirm controller/src/xmpp/xmpp_state_machine.cc 1649 2025-06-02 Mon 02:08:45:378.856 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241315604224, Pid 1126454]: Processing scm::EvSandeshSend in state Connect 2025-06-02 Mon 02:08:45:378.873 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241315604224, Pid 1126454]: SANDESH: Send FAILED: 1748830125378755 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.0.0.129 state = OpenConfirm next_state = Trigger Server Ssl Handshake event = EvStartTls identifier = cn-jenkins-deploy-platform-juju-os-542-1/dns ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-06-02 Mon 02:08:45:378.881 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241315604224, Pid 1126454]: Wrong state: Connect for event: EvSandeshSend message: XmppConnectionLog 2025-06-02 Mon 02:08:45:385.312 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241507256448, Pid 1126454]: XMPP [SYS_DEBUG]: XmppSslHandShakeMessage: XMPP Peer cn-jenkins-deploy-platform-juju-os-542-1:10.0.0.129 RECV success controller/src/xmpp/xmpp_connection.cc 402 2025-06-02 Mon 02:08:45:385.464 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241311405824, Pid 1126454]: Processing scm::EvSandeshSend in state Connect 2025-06-02 Mon 02:08:45:385.485 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241311405824, Pid 1126454]: SANDESH: Send FAILED: 1748830125385425 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.0.0.129 state = OpenConfirm next_state = OpenConfirm Feature Negotiation Success event = EvTlsHandShakeSuccess ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-06-02 Mon 02:08:45:385.494 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241311405824, Pid 1126454]: Wrong state: Connect for event: EvSandeshSend message: XmppConnectionLog 2025-06-02 Mon 02:08:45:387.521 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241307207424, Pid 1126454]: Processing scm::EvSandeshSend in state Connect 2025-06-02 Mon 02:08:45:396.045 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241307207424, Pid 1126454]: SANDESH: Send FAILED: 1748830125387483 [SYS_INFO]: XmppConnectionLog: connection= [ ip_address = 10.0.0.129 state = OpenConfirm next_state = Established event = EvXmppOpen identifier = cn-jenkins-deploy-platform-juju-os-542-1/dns ] file = controller/src/xmpp/xmpp_state_machine.cc line = 1709 2025-06-02 Mon 02:08:45:396.084 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241307207424, Pid 1126454]: Wrong state: Connect for event: EvSandeshSend message: XmppConnectionLog 2025-06-02 Mon 02:08:45:519.321 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241328199424, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:08:45:520.034 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241328199424, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating node-profile-hardware controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:08:45:700.727 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241336596224, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:08:45:702.127 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241336596224, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating node-profile-hardware controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:08:45:875.641 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241311405824, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:08:45:875.857 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241311405824, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating node-profile-hardware controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:08:46:052.491 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241290413824, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:08:46:052.744 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140240543016704, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating node-profile-hardware controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:08:46:222.754 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241315604224, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:08:46:223.007 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241294612224, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating node-profile-hardware controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:08:46:378.393 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241332397824, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:08:46:378.610 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241332397824, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating node-profile-hardware controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:08:46:564.709 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241324001024, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:08:46:564.951 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241324001024, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating node-profile-hardware controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:08:46:744.894 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241290413824, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:08:46:745.185 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241307207424, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating node-profile-hardware controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:08:46:944.947 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241303009024, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:08:46:945.131 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241303009024, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating node-profile-hardware controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:08:47:027.965 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241307207424, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:08:47:138.484 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241344993024, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:08:47:138.953 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241311405824, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating node-profile-hardware controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:08:47:287.569 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241307207424, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:08:47:287.759 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241307207424, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating node-profile-hardware controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:08:47:356.984 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241303009024, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:08:47:451.992 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241311405824, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:08:47:452.306 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241311405824, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating node-profile-hardware controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:08:47:625.225 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241349191424, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:08:47:625.486 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241349191424, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating node-profile-hardware controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:08:47:783.523 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241336596224, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:08:47:783.725 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241336596224, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating node-profile-hardware controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:08:47:955.174 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241332397824, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:08:47:955.430 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241328199424, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating node-profile-hardware controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:08:48:119.490 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241311405824, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:08:48:119.772 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241311405824, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating node-profile-hardware controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:08:48:276.161 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241307207424, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:08:48:276.355 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241307207424, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating node-profile-hardware controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:08:48:440.759 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241319802624, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:08:48:440.936 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241319802624, Pid 1126454]: IFMap [SYS_WARN]: ConfigurationMalformedReferenceWarningLog: physical_role Link name is empty node_profile b996a442-cf9b-4ff5-ad86-5ab6196a9651 controller/src/ifmap/client/config_json_parser.cc 217 2025-06-02 Mon 02:08:48:592.892 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241290413824, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:08:48:593.009 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241290413824, Pid 1126454]: IFMap [SYS_WARN]: ConfigurationMalformedReferenceWarningLog: overlay_role Link name is empty node_profile b996a442-cf9b-4ff5-ad86-5ab6196a9651 controller/src/ifmap/client/config_json_parser.cc 217 2025-06-02 Mon 02:08:48:748.112 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241332397824, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:08:48:748.277 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241332397824, Pid 1126454]: IFMap [SYS_WARN]: ConfigurationMalformedReferenceWarningLog: overlay_role Link name is empty node_profile 1dda3a7b-1747-4cd4-894a-e4811a09b749 controller/src/ifmap/client/config_json_parser.cc 217 2025-06-02 Mon 02:08:48:909.122 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140240543016704, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:08:48:909.398 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241319802624, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating node-profile-job-template controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:08:49:060.090 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241336596224, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:08:49:060.307 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241336596224, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating node-profile-job-template controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:08:49:220.665 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241344993024, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:08:49:221.702 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241349191424, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating node-profile-job-template controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:08:49:381.016 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241298810624, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:08:49:381.478 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241290413824, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating node-profile-job-template controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:08:49:533.338 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241319802624, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:08:49:533.558 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241319802624, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating node-profile-job-template controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:08:49:685.555 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241315604224, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:08:49:685.791 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140240543016704, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating node-profile-job-template controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:08:49:843.133 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241307207424, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:08:49:843.349 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241307207424, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating node-profile-job-template controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:08:49:997.836 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140240543016704, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:08:49:998.084 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241324001024, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating node-profile-job-template controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:08:50:163.367 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241324001024, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:08:50:163.534 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241324001024, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating telemetry-profile-sflow-profile controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:08:50:321.087 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241349191424, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:08:50:321.260 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241349191424, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating telemetry-profile-sflow-profile controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:08:50:475.102 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241340794624, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:08:50:475.332 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140240543016704, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating telemetry-profile-sflow-profile controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:08:50:641.288 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140240543016704, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:08:50:641.462 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140240543016704, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating telemetry-profile-grpc-profile controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:08:50:798.104 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241324001024, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:08:50:798.326 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241294612224, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating telemetry-profile-snmp-profile controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:08:50:955.171 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241332397824, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:08:50:955.324 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241332397824, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating telemetry-profile-netconf-profile controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:08:51:115.799 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241315604224, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:08:51:115.989 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241315604224, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating device-functional-group-physical-role controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:08:51:279.157 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241315604224, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:08:51:279.321 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241315604224, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating device-functional-group-physical-role controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:08:51:430.884 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241328199424, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:08:51:431.113 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241315604224, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating device-functional-group-physical-role controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:08:51:584.407 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241328199424, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:08:51:584.575 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241328199424, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating device-functional-group-physical-role controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:08:51:746.882 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241319802624, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:08:51:747.119 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241319802624, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating device-functional-group-physical-role controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:08:51:904.694 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241294612224, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:08:51:904.917 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241294612224, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating device-functional-group-physical-role controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:08:52:057.666 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241349191424, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:08:52:057.855 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241349191424, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating device-functional-group-physical-role controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:08:52:214.458 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241344993024, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:08:52:214.687 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241344993024, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating device-functional-group-physical-role controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:08:52:367.509 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241328199424, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:08:52:367.731 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241315604224, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating device-functional-group-physical-role controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:08:52:521.553 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241315604224, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:08:52:521.730 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241315604224, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating device-functional-group-physical-role controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:08:52:603.244 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140240543016704, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:08:52:603.404 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140240543016704, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating network-ipam:default-domain:default-project:_internal_ipam_ipv6_link_local controller/src/ifmap/ifmap_server_table.cc 95 2025-06-02 Mon 02:08:52:603.443 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140240543016704, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating project-network-ipam controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:08:52:893.088 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241324001024, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:08:52:893.294 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241324001024, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating virtual-network:default-domain:default-project:_internal_vn_ipv6_link_local controller/src/ifmap/ifmap_server_table.cc 95 2025-06-02 Mon 02:08:52:893.335 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241324001024, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating project-virtual-network controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:08:52:893.369 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241324001024, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapNodeOperation: Creating virtual-network-network-ipam:attr(default-domain:default-project:_internal_ipam_ipv6_link_local,default-domain:default-project:_internal_vn_ipv6_link_local) controller/src/ifmap/ifmap_server_table.cc 95 2025-06-02 Mon 02:08:52:893.383 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241324001024, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating virtual-network-network-ipam controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:08:52:893.398 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241324001024, Pid 1126454]: IFMap [SYS_DEBUG]: IFMapLinkOperation: Creating virtual-network-network-ipam controller/src/ifmap/ifmap_server_table.cc 130 2025-06-02 Mon 02:08:53:057.147 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241340794624, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:08:53:439.325 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241298810624, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:08:53:469.246 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241298810624, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:08:53:469.373 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241298810624, Pid 1126454]: IFMap [SYS_WARN]: ConfigurationMalformedReferenceWarningLog: instance_target No entry in metadata map routing_instance 18096461-c20b-4fb7-adcb-b98c95c9dc0a controller/src/ifmap/client/config_json_parser.cc 223 2025-06-02 Mon 02:09:05:529.707 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140240543016704, Pid 1126454]: Processing scm::EvSandeshSend in state Connect 2025-06-02 Mon 02:09:05:529.829 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140240543016704, Pid 1126454]: SANDESH: Send FAILED: 1748830145529275 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-juju-os-542-1:Control:contrail-dns:0 sm_queue_count = 0 sm_stats= [ ev_stats= [ [ [ event = scm::EvCollectorUpdate enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvConnectTimerExpired enqueues = 5 dequeues = 5 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvIdleHoldTimerExpired enqueues = 5 dequeues = 5 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvSandeshSend enqueues = 51 dequeues = 51 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvStart enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpDeleteSession enqueues = 5 dequeues = 5 enqueue_fails = 0 dequeue_fails = 0 ], [ event = enqueues = 68 dequeues = 68 enqueue_fails = 0 dequeue_fails = 0 ], ] ] state = Connect state_since = 1748830116547857 last_state = Connect last_event = scm::EvSandeshSend last_event_at = 1748830125387516 ] session_stats= [ num_recv_msg = 0 num_recv_msg_fail = 0 num_recv_fail = 0 num_send_msg = 0 num_send_msg_fail = 0 num_send_buffer_fail = 0 num_wait_msgq_enqueue = 0 num_wait_msgq_dequeue = 0 num_write_ready_cb_error = 0 ] session_rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] session_tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] max_sm_queue_count = 5 ] 2025-06-02 Mon 02:09:05:529.842 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140240543016704, Pid 1126454]: Wrong state: Connect for event: EvSandeshSend message: SandeshModuleClientTrace 2025-06-02 Mon 02:09:05:529.865 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140240543016704, Pid 1126454]: Processing scm::EvSandeshSend in state Connect 2025-06-02 Mon 02:09:05:530.172 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140240543016704, Pid 1126454]: SANDESH: Send FAILED: 1748830145529655 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-juju-os-542-1:Control:contrail-dns:0 client_info= [ status = Connect successful_connections = 0 pid = 1126454 http_port = 8092 start_time = 1748830000967073 collector_name = collector_ip = 10.0.0.129:8086 collector_list= [ [ (*_iter40) = 10.0.0.129:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ConfigCassInitErrorMessage [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 4 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigClientMgrDebug [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 712 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigClientMgrWarning [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 5 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigurationMalformedReferenceWarningLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 8 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = DnsOperational [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 14 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapLinkOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 454 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapNodeOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 257 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 17 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 19 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 16 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveDnsInfo [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionCreate [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionDelete [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 14 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppCreateConnection [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppSslHandShakeMessage [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppUnconsumedEvent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 1484, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 51, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-06-02 Mon 02:09:05:530.221 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140240543016704, Pid 1126454]: Wrong state: Connect for event: EvSandeshSend message: SandeshModuleClientTrace 2025-06-02 Mon 02:09:06:548.166 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241324001024, Pid 1126454]: 10.0.0.129:8086 EvConnectTimerExpired in state Connect 2025-06-02 Mon 02:09:06:548.242 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241315604224, Pid 1126454]: Processing scm::EvConnectTimerExpired in state Connect 2025-06-02 Mon 02:09:06:548.743 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241315604224, Pid 1126454]: Connect : EvConnectTimerExpired 2025-06-02 Mon 02:09:06:548.988 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241315604224, Pid 1126454]: Idle 2025-06-02 Mon 02:09:06:549.212 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241315604224, Pid 1126454]: Processing scm::EvSandeshSend in state Idle 2025-06-02 Mon 02:09:06:549.246 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241315604224, Pid 1126454]: SANDESH: Send FAILED: 1748830146548916 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-juju-os-542-1 process_status= [ [ [ module_id = contrail-dns instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvConnectTimerExpired ], [ type = Database name = Cassandra server_addrs= [ [ (*_iter6) = 10.0.0.129:9041, ] ] status = Up description = Established Cassandra connection ], [ type = Database name = RabbitMQ server_addrs= [ [ (*_iter6) = 10.0.0.129:5673, ] ] status = Up description = RabbitMQ connection established ], ] ] flag_infos= [ [ ] ] description = Collector connection down ], ] ] ] 2025-06-02 Mon 02:09:06:549.255 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241315604224, Pid 1126454]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-06-02 Mon 02:09:06:549.264 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241315604224, Pid 1126454]: Processing scm::EvTcpDeleteSession in state Idle 2025-06-02 Mon 02:09:06:549.293 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241315604224, Pid 1126454]: Processing scm::EvSandeshSend in state Idle 2025-06-02 Mon 02:09:06:549.673 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241315604224, Pid 1126454]: SANDESH: Send FAILED: 1748830146549192 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-juju-os-542-1:Control:contrail-dns:0 client_info= [ status = Idle successful_connections = 0 pid = 1126454 http_port = 8092 start_time = 1748830000967073 collector_name = collector_ip = 10.0.0.129:8086 collector_list= [ [ (*_iter40) = 10.0.0.129:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ConfigCassInitErrorMessage [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 4 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigClientMgrDebug [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 712 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigClientMgrWarning [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 5 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigurationMalformedReferenceWarningLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 8 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = DnsOperational [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 14 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapLinkOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 454 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapNodeOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 257 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 17 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 21 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 16 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveDnsInfo [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionCreate [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionDelete [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 14 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppCreateConnection [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppSslHandShakeMessage [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppUnconsumedEvent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 1484, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 53, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-06-02 Mon 02:09:06:549.716 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241315604224, Pid 1126454]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-06-02 Mon 02:09:35:530.292 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241294612224, Pid 1126454]: Processing scm::EvSandeshSend in state Idle 2025-06-02 Mon 02:09:35:530.474 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241294612224, Pid 1126454]: SANDESH: Send FAILED: 1748830175529933 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-juju-os-542-1:Control:contrail-dns:0 sm_queue_count = 0 sm_stats= [ ev_stats= [ [ [ event = scm::EvCollectorUpdate enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvConnectTimerExpired enqueues = 6 dequeues = 6 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvIdleHoldTimerExpired enqueues = 5 dequeues = 5 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvSandeshSend enqueues = 55 dequeues = 55 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvStart enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpDeleteSession enqueues = 6 dequeues = 6 enqueue_fails = 0 dequeue_fails = 0 ], [ event = enqueues = 74 dequeues = 74 enqueue_fails = 0 dequeue_fails = 0 ], ] ] state = Idle state_since = 1748830146548985 last_state = Idle last_event = scm::EvSandeshSend last_event_at = 1748830146549291 ] max_sm_queue_count = 5 ] 2025-06-02 Mon 02:09:35:530.500 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241294612224, Pid 1126454]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-06-02 Mon 02:09:35:530.516 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241294612224, Pid 1126454]: Processing scm::EvSandeshSend in state Idle 2025-06-02 Mon 02:09:35:530.881 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241294612224, Pid 1126454]: SANDESH: Send FAILED: 1748830175530138 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-juju-os-542-1:Control:contrail-dns:0 client_info= [ status = Idle successful_connections = 0 pid = 1126454 http_port = 8092 start_time = 1748830000967073 collector_name = collector_ip = 10.0.0.129:8086 collector_list= [ [ (*_iter40) = 10.0.0.129:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ConfigCassInitErrorMessage [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 4 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigClientMgrDebug [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 712 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigClientMgrWarning [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 5 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigurationMalformedReferenceWarningLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 8 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = DnsOperational [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 14 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapLinkOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 454 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapNodeOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 257 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 18 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 22 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 16 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveDnsInfo [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionCreate [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionDelete [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 14 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppCreateConnection [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppSslHandShakeMessage [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppUnconsumedEvent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 1484, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 55, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-06-02 Mon 02:09:35:530.909 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241294612224, Pid 1126454]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-06-02 Mon 02:09:36:549.163 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241303009024, Pid 1126454]: Processing scm::EvIdleHoldTimerExpired in state Idle 2025-06-02 Mon 02:09:36:549.451 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241303009024, Pid 1126454]: Connect : Start Connect timer 10.0.0.129:8086 2025-06-02 Mon 02:09:36:549.841 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241303009024, Pid 1126454]: Processing scm::EvSandeshSend in state Connect 2025-06-02 Mon 02:09:36:549.898 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241303009024, Pid 1126454]: SANDESH: Send FAILED: 1748830176549243 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-juju-os-542-1 process_status= [ [ [ module_id = contrail-dns instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.129:8086, ] ] status = Initializing description = Idle : EvIdleHoldTimerExpired -> Connect ], [ type = Database name = Cassandra server_addrs= [ [ (*_iter6) = 10.0.0.129:9041, ] ] status = Up description = Established Cassandra connection ], [ type = Database name = RabbitMQ server_addrs= [ [ (*_iter6) = 10.0.0.129:5673, ] ] status = Up description = RabbitMQ connection established ], ] ] flag_infos= [ [ ] ] description = Collector connection down ], ] ] ] 2025-06-02 Mon 02:09:36:549.924 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241303009024, Pid 1126454]: Wrong state: Connect for event: EvSandeshSend message: NodeStatusUVE 2025-06-02 Mon 02:09:36:549.938 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241303009024, Pid 1126454]: Processing scm::EvSandeshSend in state Connect 2025-06-02 Mon 02:09:36:550.365 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241303009024, Pid 1126454]: SANDESH: Send FAILED: 1748830176549713 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-juju-os-542-1:Control:contrail-dns:0 client_info= [ status = Connect successful_connections = 0 pid = 1126454 http_port = 8092 start_time = 1748830000967073 collector_name = collector_ip = 10.0.0.129:8086 collector_list= [ [ (*_iter40) = 10.0.0.129:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ConfigCassInitErrorMessage [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 4 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigClientMgrDebug [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 712 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigClientMgrWarning [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 5 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigurationMalformedReferenceWarningLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 8 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = DnsOperational [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 14 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapLinkOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 454 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapNodeOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 257 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 18 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 24 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 16 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveDnsInfo [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionCreate [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionDelete [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 14 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppCreateConnection [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppSslHandShakeMessage [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppUnconsumedEvent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 1484, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 57, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-06-02 Mon 02:09:36:550.396 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241303009024, Pid 1126454]: Wrong state: Connect for event: EvSandeshSend message: SandeshModuleClientTrace 2025-06-02 Mon 02:10:05:530.479 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241294612224, Pid 1126454]: Processing scm::EvSandeshSend in state Connect 2025-06-02 Mon 02:10:05:530.747 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241294612224, Pid 1126454]: SANDESH: Send FAILED: 1748830205530327 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-juju-os-542-1:Control:contrail-dns:0 sm_queue_count = 0 sm_stats= [ ev_stats= [ [ [ event = scm::EvCollectorUpdate enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvConnectTimerExpired enqueues = 6 dequeues = 6 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvIdleHoldTimerExpired enqueues = 6 dequeues = 6 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvSandeshSend enqueues = 59 dequeues = 59 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvStart enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpDeleteSession enqueues = 6 dequeues = 6 enqueue_fails = 0 dequeue_fails = 0 ], [ event = enqueues = 79 dequeues = 79 enqueue_fails = 0 dequeue_fails = 0 ], ] ] state = Connect state_since = 1748830176549251 last_state = Connect last_event = scm::EvSandeshSend last_event_at = 1748830176549935 ] session_stats= [ num_recv_msg = 0 num_recv_msg_fail = 0 num_recv_fail = 0 num_send_msg = 0 num_send_msg_fail = 0 num_send_buffer_fail = 0 num_wait_msgq_enqueue = 0 num_wait_msgq_dequeue = 0 num_write_ready_cb_error = 0 ] session_rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] session_tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] max_sm_queue_count = 5 ] 2025-06-02 Mon 02:10:05:530.767 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241294612224, Pid 1126454]: Wrong state: Connect for event: EvSandeshSend message: SandeshModuleClientTrace 2025-06-02 Mon 02:10:05:530.831 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241294612224, Pid 1126454]: Processing scm::EvSandeshSend in state Connect 2025-06-02 Mon 02:10:05:531.403 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241294612224, Pid 1126454]: SANDESH: Send FAILED: 1748830205530434 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-juju-os-542-1:Control:contrail-dns:0 client_info= [ status = Connect successful_connections = 0 pid = 1126454 http_port = 8092 start_time = 1748830000967073 collector_name = collector_ip = 10.0.0.129:8086 collector_list= [ [ (*_iter40) = 10.0.0.129:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ConfigCassInitErrorMessage [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 4 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigClientMgrDebug [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 712 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigClientMgrWarning [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 5 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigurationMalformedReferenceWarningLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 8 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = DnsOperational [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 14 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapLinkOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 454 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapNodeOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 257 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 19 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 25 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 16 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveDnsInfo [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionCreate [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionDelete [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 14 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppCreateConnection [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppSslHandShakeMessage [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppUnconsumedEvent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 1484, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 59, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-06-02 Mon 02:10:05:531.440 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241294612224, Pid 1126454]: Wrong state: Connect for event: EvSandeshSend message: SandeshModuleClientTrace 2025-06-02 Mon 02:10:06:549.625 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241332397824, Pid 1126454]: 10.0.0.129:8086 EvConnectTimerExpired in state Connect 2025-06-02 Mon 02:10:06:550.284 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241349191424, Pid 1126454]: Processing scm::EvConnectTimerExpired in state Connect 2025-06-02 Mon 02:10:06:550.323 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241349191424, Pid 1126454]: Connect : EvConnectTimerExpired 2025-06-02 Mon 02:10:06:550.527 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241349191424, Pid 1126454]: Idle 2025-06-02 Mon 02:10:06:550.801 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241349191424, Pid 1126454]: Processing scm::EvSandeshSend in state Idle 2025-06-02 Mon 02:10:06:550.852 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241349191424, Pid 1126454]: SANDESH: Send FAILED: 1748830206550355 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-juju-os-542-1 process_status= [ [ [ module_id = contrail-dns instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvConnectTimerExpired ], [ type = Database name = Cassandra server_addrs= [ [ (*_iter6) = 10.0.0.129:9041, ] ] status = Up description = Established Cassandra connection ], [ type = Database name = RabbitMQ server_addrs= [ [ (*_iter6) = 10.0.0.129:5673, ] ] status = Up description = RabbitMQ connection established ], ] ] flag_infos= [ [ ] ] description = Collector connection down ], ] ] ] 2025-06-02 Mon 02:10:06:550.865 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241349191424, Pid 1126454]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-06-02 Mon 02:10:06:550.877 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241349191424, Pid 1126454]: Processing scm::EvTcpDeleteSession in state Idle 2025-06-02 Mon 02:10:06:550.917 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241349191424, Pid 1126454]: Processing scm::EvSandeshSend in state Idle 2025-06-02 Mon 02:10:06:551.311 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241349191424, Pid 1126454]: SANDESH: Send FAILED: 1748830206550775 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-juju-os-542-1:Control:contrail-dns:0 client_info= [ status = Idle successful_connections = 0 pid = 1126454 http_port = 8092 start_time = 1748830000967073 collector_name = collector_ip = 10.0.0.129:8086 collector_list= [ [ (*_iter40) = 10.0.0.129:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ConfigCassInitErrorMessage [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 4 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigClientMgrDebug [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 712 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigClientMgrWarning [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 5 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigurationMalformedReferenceWarningLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 8 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = DnsOperational [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 14 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapLinkOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 454 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapNodeOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 257 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 19 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 27 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 16 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveDnsInfo [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionCreate [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionDelete [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 14 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppCreateConnection [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppSslHandShakeMessage [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppUnconsumedEvent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 1484, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 61, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-06-02 Mon 02:10:06:551.417 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241349191424, Pid 1126454]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-06-02 Mon 02:10:35:530.743 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241344993024, Pid 1126454]: Processing scm::EvSandeshSend in state Idle 2025-06-02 Mon 02:10:35:530.846 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241344993024, Pid 1126454]: SANDESH: Send FAILED: 1748830235530567 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-juju-os-542-1:Control:contrail-dns:0 sm_queue_count = 0 sm_stats= [ ev_stats= [ [ [ event = scm::EvCollectorUpdate enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvConnectTimerExpired enqueues = 7 dequeues = 7 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvIdleHoldTimerExpired enqueues = 6 dequeues = 6 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvSandeshSend enqueues = 63 dequeues = 63 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvStart enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpDeleteSession enqueues = 7 dequeues = 7 enqueue_fails = 0 dequeue_fails = 0 ], [ event = enqueues = 85 dequeues = 85 enqueue_fails = 0 dequeue_fails = 0 ], ] ] state = Idle state_since = 1748830206550524 last_state = Idle last_event = scm::EvSandeshSend last_event_at = 1748830206550915 ] max_sm_queue_count = 5 ] 2025-06-02 Mon 02:10:35:530.861 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241344993024, Pid 1126454]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-06-02 Mon 02:10:35:530.872 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241344993024, Pid 1126454]: Processing scm::EvSandeshSend in state Idle 2025-06-02 Mon 02:10:35:531.163 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241344993024, Pid 1126454]: SANDESH: Send FAILED: 1748830235530695 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-juju-os-542-1:Control:contrail-dns:0 client_info= [ status = Idle successful_connections = 0 pid = 1126454 http_port = 8092 start_time = 1748830000967073 collector_name = collector_ip = 10.0.0.129:8086 collector_list= [ [ (*_iter40) = 10.0.0.129:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ConfigCassInitErrorMessage [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 4 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigClientMgrDebug [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 712 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigClientMgrWarning [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 5 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigurationMalformedReferenceWarningLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 8 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = DnsOperational [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 14 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapLinkOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 454 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapNodeOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 257 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 20 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 28 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 16 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveDnsInfo [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionCreate [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionDelete [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 14 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppCreateConnection [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppSslHandShakeMessage [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppUnconsumedEvent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 1484, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 63, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-06-02 Mon 02:10:35:531.187 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241344993024, Pid 1126454]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-06-02 Mon 02:10:36:550.678 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241324001024, Pid 1126454]: Processing scm::EvIdleHoldTimerExpired in state Idle 2025-06-02 Mon 02:10:36:550.969 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241324001024, Pid 1126454]: Connect : Start Connect timer 10.0.0.129:8086 2025-06-02 Mon 02:10:36:551.125 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241324001024, Pid 1126454]: Processing scm::EvSandeshSend in state Connect 2025-06-02 Mon 02:10:36:551.156 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241324001024, Pid 1126454]: SANDESH: Send FAILED: 1748830236550791 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-juju-os-542-1 process_status= [ [ [ module_id = contrail-dns instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.129:8086, ] ] status = Initializing description = Idle : EvIdleHoldTimerExpired -> Connect ], [ type = Database name = Cassandra server_addrs= [ [ (*_iter6) = 10.0.0.129:9041, ] ] status = Up description = Established Cassandra connection ], [ type = Database name = RabbitMQ server_addrs= [ [ (*_iter6) = 10.0.0.129:5673, ] ] status = Up description = RabbitMQ connection established ], ] ] flag_infos= [ [ ] ] description = Collector connection down ], ] ] ] 2025-06-02 Mon 02:10:36:551.174 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241324001024, Pid 1126454]: Wrong state: Connect for event: EvSandeshSend message: NodeStatusUVE 2025-06-02 Mon 02:10:36:551.183 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241324001024, Pid 1126454]: Processing scm::EvSandeshSend in state Connect 2025-06-02 Mon 02:10:36:551.537 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241324001024, Pid 1126454]: SANDESH: Send FAILED: 1748830236551105 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-juju-os-542-1:Control:contrail-dns:0 client_info= [ status = Connect successful_connections = 0 pid = 1126454 http_port = 8092 start_time = 1748830000967073 collector_name = collector_ip = 10.0.0.129:8086 collector_list= [ [ (*_iter40) = 10.0.0.129:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ConfigCassInitErrorMessage [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 4 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigClientMgrDebug [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 712 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigClientMgrWarning [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 5 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigurationMalformedReferenceWarningLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 8 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = DnsOperational [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 14 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapLinkOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 454 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapNodeOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 257 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 20 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 30 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 16 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveDnsInfo [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionCreate [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionDelete [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 14 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppCreateConnection [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppSslHandShakeMessage [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppUnconsumedEvent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 1484, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 65, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-06-02 Mon 02:10:36:551.594 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241324001024, Pid 1126454]: Wrong state: Connect for event: EvSandeshSend message: SandeshModuleClientTrace 2025-06-02 Mon 02:11:00:232.555 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241311405824, Pid 1126454]: ConfigClientManager [SYS_DEBUG]: ConfigClientMgrDebug: Cassandra SM: Established Cassandra connection src/contrail-common/config-client-mgr/config_cassandra_client.cc 573 2025-06-02 Mon 02:11:05:531.055 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241336596224, Pid 1126454]: Processing scm::EvSandeshSend in state Connect 2025-06-02 Mon 02:11:05:531.169 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241336596224, Pid 1126454]: SANDESH: Send FAILED: 1748830265530898 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-juju-os-542-1:Control:contrail-dns:0 sm_queue_count = 0 sm_stats= [ ev_stats= [ [ [ event = scm::EvCollectorUpdate enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvConnectTimerExpired enqueues = 7 dequeues = 7 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvIdleHoldTimerExpired enqueues = 7 dequeues = 7 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvSandeshSend enqueues = 67 dequeues = 67 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvStart enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpDeleteSession enqueues = 7 dequeues = 7 enqueue_fails = 0 dequeue_fails = 0 ], [ event = enqueues = 90 dequeues = 90 enqueue_fails = 0 dequeue_fails = 0 ], ] ] state = Connect state_since = 1748830236550797 last_state = Connect last_event = scm::EvSandeshSend last_event_at = 1748830236551181 ] session_stats= [ num_recv_msg = 0 num_recv_msg_fail = 0 num_recv_fail = 0 num_send_msg = 0 num_send_msg_fail = 0 num_send_buffer_fail = 0 num_wait_msgq_enqueue = 0 num_wait_msgq_dequeue = 0 num_write_ready_cb_error = 0 ] session_rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] session_tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] max_sm_queue_count = 5 ] 2025-06-02 Mon 02:11:05:531.194 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241336596224, Pid 1126454]: Wrong state: Connect for event: EvSandeshSend message: SandeshModuleClientTrace 2025-06-02 Mon 02:11:05:531.204 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241336596224, Pid 1126454]: Processing scm::EvSandeshSend in state Connect 2025-06-02 Mon 02:11:05:531.534 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241336596224, Pid 1126454]: SANDESH: Send FAILED: 1748830265531018 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-juju-os-542-1:Control:contrail-dns:0 client_info= [ status = Connect successful_connections = 0 pid = 1126454 http_port = 8092 start_time = 1748830000967073 collector_name = collector_ip = 10.0.0.129:8086 collector_list= [ [ (*_iter40) = 10.0.0.129:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ConfigCassInitErrorMessage [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 4 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigClientMgrDebug [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 713 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigClientMgrWarning [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 5 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigurationMalformedReferenceWarningLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 8 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = DnsOperational [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 14 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapLinkOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 454 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapNodeOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 257 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 21 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 31 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 16 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveDnsInfo [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionCreate [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionDelete [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 14 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppCreateConnection [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppSslHandShakeMessage [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppUnconsumedEvent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 1485, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 67, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-06-02 Mon 02:11:05:531.561 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241336596224, Pid 1126454]: Wrong state: Connect for event: EvSandeshSend message: SandeshModuleClientTrace 2025-06-02 Mon 02:11:06:551.114 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241344993024, Pid 1126454]: 10.0.0.129:8086 EvConnectTimerExpired in state Connect 2025-06-02 Mon 02:11:06:551.200 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241294612224, Pid 1126454]: Processing scm::EvConnectTimerExpired in state Connect 2025-06-02 Mon 02:11:06:551.236 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241294612224, Pid 1126454]: Connect : EvConnectTimerExpired 2025-06-02 Mon 02:11:06:551.317 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241294612224, Pid 1126454]: Idle 2025-06-02 Mon 02:11:06:551.417 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241294612224, Pid 1126454]: Processing scm::EvSandeshSend in state Idle 2025-06-02 Mon 02:11:06:551.447 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241294612224, Pid 1126454]: SANDESH: Send FAILED: 1748830266551268 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-juju-os-542-1 process_status= [ [ [ module_id = contrail-dns instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvConnectTimerExpired ], [ type = Database name = Cassandra server_addrs= [ [ (*_iter6) = 10.0.0.129:9041, ] ] status = Up description = Established Cassandra connection ], [ type = Database name = RabbitMQ server_addrs= [ [ (*_iter6) = 10.0.0.129:5673, ] ] status = Up description = RabbitMQ connection established ], ] ] flag_infos= [ [ ] ] description = Collector connection down ], ] ] ] 2025-06-02 Mon 02:11:06:551.768 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241294612224, Pid 1126454]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-06-02 Mon 02:11:06:551.799 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241294612224, Pid 1126454]: Processing scm::EvTcpDeleteSession in state Idle 2025-06-02 Mon 02:11:06:551.834 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241294612224, Pid 1126454]: Processing scm::EvSandeshSend in state Idle 2025-06-02 Mon 02:11:06:552.130 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241294612224, Pid 1126454]: SANDESH: Send FAILED: 1748830266551404 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-juju-os-542-1:Control:contrail-dns:0 client_info= [ status = Idle successful_connections = 0 pid = 1126454 http_port = 8092 start_time = 1748830000967073 collector_name = collector_ip = 10.0.0.129:8086 collector_list= [ [ (*_iter40) = 10.0.0.129:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ConfigCassInitErrorMessage [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 4 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigClientMgrDebug [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 713 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigClientMgrWarning [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 5 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigurationMalformedReferenceWarningLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 8 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = DnsOperational [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 14 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapLinkOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 454 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapNodeOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 257 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 21 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 33 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 16 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveDnsInfo [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionCreate [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionDelete [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 14 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppCreateConnection [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppSslHandShakeMessage [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppUnconsumedEvent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 1485, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 69, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-06-02 Mon 02:11:06:552.162 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241294612224, Pid 1126454]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-06-02 Mon 02:11:35:531.328 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241349191424, Pid 1126454]: Processing scm::EvSandeshSend in state Idle 2025-06-02 Mon 02:11:35:531.426 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241349191424, Pid 1126454]: SANDESH: Send FAILED: 1748830295531152 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-juju-os-542-1:Control:contrail-dns:0 sm_queue_count = 0 sm_stats= [ ev_stats= [ [ [ event = scm::EvCollectorUpdate enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvConnectTimerExpired enqueues = 8 dequeues = 8 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvIdleHoldTimerExpired enqueues = 7 dequeues = 7 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvSandeshSend enqueues = 71 dequeues = 71 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvStart enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpDeleteSession enqueues = 8 dequeues = 8 enqueue_fails = 0 dequeue_fails = 0 ], [ event = enqueues = 96 dequeues = 96 enqueue_fails = 0 dequeue_fails = 0 ], ] ] state = Idle state_since = 1748830266551315 last_state = Idle last_event = scm::EvSandeshSend last_event_at = 1748830266551831 ] max_sm_queue_count = 5 ] 2025-06-02 Mon 02:11:35:531.440 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241349191424, Pid 1126454]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-06-02 Mon 02:11:35:531.468 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241349191424, Pid 1126454]: Processing scm::EvSandeshSend in state Idle 2025-06-02 Mon 02:11:35:531.789 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241349191424, Pid 1126454]: SANDESH: Send FAILED: 1748830295531280 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-juju-os-542-1:Control:contrail-dns:0 client_info= [ status = Idle successful_connections = 0 pid = 1126454 http_port = 8092 start_time = 1748830000967073 collector_name = collector_ip = 10.0.0.129:8086 collector_list= [ [ (*_iter40) = 10.0.0.129:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ConfigCassInitErrorMessage [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 4 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigClientMgrDebug [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 713 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigClientMgrWarning [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 5 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigurationMalformedReferenceWarningLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 8 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = DnsOperational [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 14 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapLinkOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 454 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapNodeOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 257 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 22 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 34 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 16 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveDnsInfo [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionCreate [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionDelete [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 14 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppCreateConnection [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppSslHandShakeMessage [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppUnconsumedEvent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 1485, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 71, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-06-02 Mon 02:11:35:531.820 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241349191424, Pid 1126454]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-06-02 Mon 02:11:36:551.477 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241332397824, Pid 1126454]: Processing scm::EvIdleHoldTimerExpired in state Idle 2025-06-02 Mon 02:11:36:551.935 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241332397824, Pid 1126454]: Connect : Start Connect timer 10.0.0.129:8086 2025-06-02 Mon 02:11:36:552.157 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241332397824, Pid 1126454]: Processing scm::EvSandeshSend in state Connect 2025-06-02 Mon 02:11:36:552.207 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241332397824, Pid 1126454]: SANDESH: Send FAILED: 1748830296551559 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-juju-os-542-1 process_status= [ [ [ module_id = contrail-dns instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.129:8086, ] ] status = Initializing description = Idle : EvIdleHoldTimerExpired -> Connect ], [ type = Database name = Cassandra server_addrs= [ [ (*_iter6) = 10.0.0.129:9041, ] ] status = Up description = Established Cassandra connection ], [ type = Database name = RabbitMQ server_addrs= [ [ (*_iter6) = 10.0.0.129:5673, ] ] status = Up description = RabbitMQ connection established ], ] ] flag_infos= [ [ ] ] description = Collector connection down ], ] ] ] 2025-06-02 Mon 02:11:36:552.233 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241332397824, Pid 1126454]: Wrong state: Connect for event: EvSandeshSend message: NodeStatusUVE 2025-06-02 Mon 02:11:36:552.247 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241332397824, Pid 1126454]: Processing scm::EvSandeshSend in state Connect 2025-06-02 Mon 02:11:36:552.756 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241332397824, Pid 1126454]: SANDESH: Send FAILED: 1748830296552125 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-juju-os-542-1:Control:contrail-dns:0 client_info= [ status = Connect successful_connections = 0 pid = 1126454 http_port = 8092 start_time = 1748830000967073 collector_name = collector_ip = 10.0.0.129:8086 collector_list= [ [ (*_iter40) = 10.0.0.129:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ConfigCassInitErrorMessage [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 4 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigClientMgrDebug [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 713 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigClientMgrWarning [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 5 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigurationMalformedReferenceWarningLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 8 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = DnsOperational [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 14 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapLinkOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 454 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapNodeOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 257 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 22 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 36 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 16 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveDnsInfo [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionCreate [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionDelete [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 14 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppCreateConnection [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppSslHandShakeMessage [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppUnconsumedEvent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 1485, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 73, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-06-02 Mon 02:11:36:552.845 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241332397824, Pid 1126454]: Wrong state: Connect for event: EvSandeshSend message: SandeshModuleClientTrace 2025-06-02 Mon 02:12:05:531.893 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241311405824, Pid 1126454]: Processing scm::EvSandeshSend in state Connect 2025-06-02 Mon 02:12:05:532.246 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241311405824, Pid 1126454]: SANDESH: Send FAILED: 1748830325531470 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-juju-os-542-1:Control:contrail-dns:0 sm_queue_count = 0 sm_stats= [ ev_stats= [ [ [ event = scm::EvCollectorUpdate enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvConnectTimerExpired enqueues = 8 dequeues = 8 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvIdleHoldTimerExpired enqueues = 8 dequeues = 8 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvSandeshSend enqueues = 75 dequeues = 75 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvStart enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpDeleteSession enqueues = 8 dequeues = 8 enqueue_fails = 0 dequeue_fails = 0 ], [ event = enqueues = 101 dequeues = 101 enqueue_fails = 0 dequeue_fails = 0 ], ] ] state = Connect state_since = 1748830296551570 last_state = Connect last_event = scm::EvSandeshSend last_event_at = 1748830296552245 ] session_stats= [ num_recv_msg = 0 num_recv_msg_fail = 0 num_recv_fail = 0 num_send_msg = 0 num_send_msg_fail = 0 num_send_buffer_fail = 0 num_wait_msgq_enqueue = 0 num_wait_msgq_dequeue = 0 num_write_ready_cb_error = 0 ] session_rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] session_tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] max_sm_queue_count = 5 ] 2025-06-02 Mon 02:12:05:532.289 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241311405824, Pid 1126454]: Wrong state: Connect for event: EvSandeshSend message: SandeshModuleClientTrace 2025-06-02 Mon 02:12:05:532.306 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241311405824, Pid 1126454]: Processing scm::EvSandeshSend in state Connect 2025-06-02 Mon 02:12:05:532.573 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241311405824, Pid 1126454]: SANDESH: Send FAILED: 1748830325531820 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-juju-os-542-1:Control:contrail-dns:0 client_info= [ status = Connect successful_connections = 0 pid = 1126454 http_port = 8092 start_time = 1748830000967073 collector_name = collector_ip = 10.0.0.129:8086 collector_list= [ [ (*_iter40) = 10.0.0.129:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ConfigCassInitErrorMessage [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 4 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigClientMgrDebug [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 713 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigClientMgrWarning [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 5 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigurationMalformedReferenceWarningLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 8 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = DnsOperational [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 14 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapLinkOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 454 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapNodeOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 257 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 23 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 37 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 16 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveDnsInfo [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionCreate [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionDelete [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 14 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppCreateConnection [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppSslHandShakeMessage [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppUnconsumedEvent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 1485, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 75, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-06-02 Mon 02:12:05:532.592 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241311405824, Pid 1126454]: Wrong state: Connect for event: EvSandeshSend message: SandeshModuleClientTrace 2025-06-02 Mon 02:12:06:552.158 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241303009024, Pid 1126454]: 10.0.0.129:8086 EvConnectTimerExpired in state Connect 2025-06-02 Mon 02:12:06:552.214 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241303009024, Pid 1126454]: Processing scm::EvConnectTimerExpired in state Connect 2025-06-02 Mon 02:12:06:552.224 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241303009024, Pid 1126454]: Connect : EvConnectTimerExpired 2025-06-02 Mon 02:12:06:552.302 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241303009024, Pid 1126454]: Idle 2025-06-02 Mon 02:12:06:552.429 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241303009024, Pid 1126454]: Processing scm::EvSandeshSend in state Idle 2025-06-02 Mon 02:12:06:552.470 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241303009024, Pid 1126454]: SANDESH: Send FAILED: 1748830326552252 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-juju-os-542-1 process_status= [ [ [ module_id = contrail-dns instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvConnectTimerExpired ], [ type = Database name = Cassandra server_addrs= [ [ (*_iter6) = 10.0.0.129:9041, ] ] status = Up description = Established Cassandra connection ], [ type = Database name = RabbitMQ server_addrs= [ [ (*_iter6) = 10.0.0.129:5673, ] ] status = Up description = RabbitMQ connection established ], ] ] flag_infos= [ [ ] ] description = Collector connection down ], ] ] ] 2025-06-02 Mon 02:12:06:552.483 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241303009024, Pid 1126454]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-06-02 Mon 02:12:06:552.494 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241303009024, Pid 1126454]: Processing scm::EvTcpDeleteSession in state Idle 2025-06-02 Mon 02:12:06:552.507 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241303009024, Pid 1126454]: Processing scm::EvSandeshSend in state Idle 2025-06-02 Mon 02:12:06:552.958 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241303009024, Pid 1126454]: SANDESH: Send FAILED: 1748830326552411 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-juju-os-542-1:Control:contrail-dns:0 client_info= [ status = Idle successful_connections = 0 pid = 1126454 http_port = 8092 start_time = 1748830000967073 collector_name = collector_ip = 10.0.0.129:8086 collector_list= [ [ (*_iter40) = 10.0.0.129:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ConfigCassInitErrorMessage [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 4 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigClientMgrDebug [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 713 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigClientMgrWarning [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 5 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigurationMalformedReferenceWarningLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 8 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = DnsOperational [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 14 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapLinkOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 454 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapNodeOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 257 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 23 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 39 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 16 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveDnsInfo [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionCreate [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionDelete [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 14 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppCreateConnection [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppSslHandShakeMessage [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppUnconsumedEvent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 1485, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 77, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-06-02 Mon 02:12:06:553.007 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241303009024, Pid 1126454]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-06-02 Mon 02:12:35:532.205 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241307207424, Pid 1126454]: Processing scm::EvSandeshSend in state Idle 2025-06-02 Mon 02:12:35:532.330 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241307207424, Pid 1126454]: SANDESH: Send FAILED: 1748830355531973 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-juju-os-542-1:Control:contrail-dns:0 sm_queue_count = 0 sm_stats= [ ev_stats= [ [ [ event = scm::EvCollectorUpdate enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvConnectTimerExpired enqueues = 9 dequeues = 9 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvIdleHoldTimerExpired enqueues = 8 dequeues = 8 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvSandeshSend enqueues = 79 dequeues = 79 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvStart enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpDeleteSession enqueues = 9 dequeues = 9 enqueue_fails = 0 dequeue_fails = 0 ], [ event = enqueues = 107 dequeues = 107 enqueue_fails = 0 dequeue_fails = 0 ], ] ] state = Idle state_since = 1748830326552300 last_state = Idle last_event = scm::EvSandeshSend last_event_at = 1748830326552505 ] max_sm_queue_count = 5 ] 2025-06-02 Mon 02:12:35:532.348 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241307207424, Pid 1126454]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-06-02 Mon 02:12:35:532.359 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241307207424, Pid 1126454]: Processing scm::EvSandeshSend in state Idle 2025-06-02 Mon 02:12:35:532.744 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241307207424, Pid 1126454]: SANDESH: Send FAILED: 1748830355532159 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-juju-os-542-1:Control:contrail-dns:0 client_info= [ status = Idle successful_connections = 0 pid = 1126454 http_port = 8092 start_time = 1748830000967073 collector_name = collector_ip = 10.0.0.129:8086 collector_list= [ [ (*_iter40) = 10.0.0.129:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ConfigCassInitErrorMessage [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 4 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigClientMgrDebug [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 713 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigClientMgrWarning [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 5 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigurationMalformedReferenceWarningLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 8 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = DnsOperational [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 14 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapLinkOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 454 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapNodeOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 257 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 24 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 40 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 16 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveDnsInfo [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionCreate [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionDelete [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 14 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppCreateConnection [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppSslHandShakeMessage [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppUnconsumedEvent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 1485, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 79, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-06-02 Mon 02:12:35:532.797 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241307207424, Pid 1126454]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-06-02 Mon 02:12:36:552.570 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241315604224, Pid 1126454]: Processing scm::EvIdleHoldTimerExpired in state Idle 2025-06-02 Mon 02:12:36:552.887 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241315604224, Pid 1126454]: Connect : Start Connect timer 10.0.0.129:8086 2025-06-02 Mon 02:12:36:553.175 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241315604224, Pid 1126454]: Processing scm::EvSandeshSend in state Connect 2025-06-02 Mon 02:12:36:553.226 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241315604224, Pid 1126454]: SANDESH: Send FAILED: 1748830356552676 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-juju-os-542-1 process_status= [ [ [ module_id = contrail-dns instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.129:8086, ] ] status = Initializing description = Idle : EvIdleHoldTimerExpired -> Connect ], [ type = Database name = Cassandra server_addrs= [ [ (*_iter6) = 10.0.0.129:9041, ] ] status = Up description = Established Cassandra connection ], [ type = Database name = RabbitMQ server_addrs= [ [ (*_iter6) = 10.0.0.129:5673, ] ] status = Up description = RabbitMQ connection established ], ] ] flag_infos= [ [ ] ] description = Collector connection down ], ] ] ] 2025-06-02 Mon 02:12:36:553.251 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241315604224, Pid 1126454]: Wrong state: Connect for event: EvSandeshSend message: NodeStatusUVE 2025-06-02 Mon 02:12:36:553.265 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241315604224, Pid 1126454]: Processing scm::EvSandeshSend in state Connect 2025-06-02 Mon 02:12:36:553.741 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241315604224, Pid 1126454]: SANDESH: Send FAILED: 1748830356553037 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-juju-os-542-1:Control:contrail-dns:0 client_info= [ status = Connect successful_connections = 0 pid = 1126454 http_port = 8092 start_time = 1748830000967073 collector_name = collector_ip = 10.0.0.129:8086 collector_list= [ [ (*_iter40) = 10.0.0.129:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ConfigCassInitErrorMessage [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 4 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigClientMgrDebug [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 713 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigClientMgrWarning [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 5 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigurationMalformedReferenceWarningLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 8 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = DnsOperational [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 14 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapLinkOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 454 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapNodeOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 257 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 24 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 42 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 16 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveDnsInfo [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionCreate [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionDelete [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 14 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppCreateConnection [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppSslHandShakeMessage [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppUnconsumedEvent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 1485, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 81, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-06-02 Mon 02:12:36:553.774 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241315604224, Pid 1126454]: Wrong state: Connect for event: EvSandeshSend message: SandeshModuleClientTrace 2025-06-02 Mon 02:13:05:532.472 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241349191424, Pid 1126454]: Processing scm::EvSandeshSend in state Connect 2025-06-02 Mon 02:13:05:532.594 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241349191424, Pid 1126454]: SANDESH: Send FAILED: 1748830385532335 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-juju-os-542-1:Control:contrail-dns:0 sm_queue_count = 0 sm_stats= [ ev_stats= [ [ [ event = scm::EvCollectorUpdate enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvConnectTimerExpired enqueues = 9 dequeues = 9 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvIdleHoldTimerExpired enqueues = 9 dequeues = 9 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvSandeshSend enqueues = 83 dequeues = 83 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvStart enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpDeleteSession enqueues = 9 dequeues = 9 enqueue_fails = 0 dequeue_fails = 0 ], [ event = enqueues = 112 dequeues = 112 enqueue_fails = 0 dequeue_fails = 0 ], ] ] state = Connect state_since = 1748830356552684 last_state = Connect last_event = scm::EvSandeshSend last_event_at = 1748830356553263 ] session_stats= [ num_recv_msg = 0 num_recv_msg_fail = 0 num_recv_fail = 0 num_send_msg = 0 num_send_msg_fail = 0 num_send_buffer_fail = 0 num_wait_msgq_enqueue = 0 num_wait_msgq_dequeue = 0 num_write_ready_cb_error = 0 ] session_rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] session_tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] max_sm_queue_count = 5 ] 2025-06-02 Mon 02:13:05:532.647 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241349191424, Pid 1126454]: Wrong state: Connect for event: EvSandeshSend message: SandeshModuleClientTrace 2025-06-02 Mon 02:13:05:532.670 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241349191424, Pid 1126454]: Processing scm::EvSandeshSend in state Connect 2025-06-02 Mon 02:13:05:533.222 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241349191424, Pid 1126454]: SANDESH: Send FAILED: 1748830385532425 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-juju-os-542-1:Control:contrail-dns:0 client_info= [ status = Connect successful_connections = 0 pid = 1126454 http_port = 8092 start_time = 1748830000967073 collector_name = collector_ip = 10.0.0.129:8086 collector_list= [ [ (*_iter40) = 10.0.0.129:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ConfigCassInitErrorMessage [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 4 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigClientMgrDebug [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 713 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigClientMgrWarning [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 5 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigurationMalformedReferenceWarningLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 8 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = DnsOperational [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 14 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapLinkOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 454 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapNodeOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 257 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 25 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 43 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 16 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveDnsInfo [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionCreate [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionDelete [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 14 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppCreateConnection [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppSslHandShakeMessage [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppUnconsumedEvent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 1485, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 83, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-06-02 Mon 02:13:05:533.270 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241349191424, Pid 1126454]: Wrong state: Connect for event: EvSandeshSend message: SandeshModuleClientTrace 2025-06-02 Mon 02:13:06:553.028 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241303009024, Pid 1126454]: 10.0.0.129:8086 EvConnectTimerExpired in state Connect 2025-06-02 Mon 02:13:06:553.106 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241315604224, Pid 1126454]: Processing scm::EvConnectTimerExpired in state Connect 2025-06-02 Mon 02:13:06:553.144 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241315604224, Pid 1126454]: Connect : EvConnectTimerExpired 2025-06-02 Mon 02:13:06:553.235 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241315604224, Pid 1126454]: Idle 2025-06-02 Mon 02:13:06:553.351 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241315604224, Pid 1126454]: Processing scm::EvSandeshSend in state Idle 2025-06-02 Mon 02:13:06:553.389 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241315604224, Pid 1126454]: SANDESH: Send FAILED: 1748830386553176 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-juju-os-542-1 process_status= [ [ [ module_id = contrail-dns instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvConnectTimerExpired ], [ type = Database name = Cassandra server_addrs= [ [ (*_iter6) = 10.0.0.129:9041, ] ] status = Up description = Established Cassandra connection ], [ type = Database name = RabbitMQ server_addrs= [ [ (*_iter6) = 10.0.0.129:5673, ] ] status = Up description = RabbitMQ connection established ], ] ] flag_infos= [ [ ] ] description = Collector connection down ], ] ] ] 2025-06-02 Mon 02:13:06:553.399 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241315604224, Pid 1126454]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-06-02 Mon 02:13:06:553.407 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241315604224, Pid 1126454]: Processing scm::EvTcpDeleteSession in state Idle 2025-06-02 Mon 02:13:06:553.435 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241315604224, Pid 1126454]: Processing scm::EvSandeshSend in state Idle 2025-06-02 Mon 02:13:06:553.812 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241315604224, Pid 1126454]: SANDESH: Send FAILED: 1748830386553334 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-juju-os-542-1:Control:contrail-dns:0 client_info= [ status = Idle successful_connections = 0 pid = 1126454 http_port = 8092 start_time = 1748830000967073 collector_name = collector_ip = 10.0.0.129:8086 collector_list= [ [ (*_iter40) = 10.0.0.129:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ConfigCassInitErrorMessage [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 4 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigClientMgrDebug [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 713 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigClientMgrWarning [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 5 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigurationMalformedReferenceWarningLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 8 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = DnsOperational [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 14 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapLinkOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 454 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapNodeOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 257 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 25 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 45 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 16 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveDnsInfo [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionCreate [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionDelete [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 14 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppCreateConnection [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppSslHandShakeMessage [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppUnconsumedEvent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 1485, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 85, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-06-02 Mon 02:13:06:553.856 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241315604224, Pid 1126454]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-06-02 Mon 02:13:09:332.275 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241507256448, Pid 1126454]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.0.0.129:8092::10.0.0.129:51044 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 256 2025-06-02 Mon 02:13:09:378.423 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241507256448, Pid 1126454]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.0.0.129:8092::10.0.0.129:51044 < Read failed due to error asio.ssl 335544539 : short read src/contrail-common/io/tcp_session.cc 504 2025-06-02 Mon 02:13:10:774.385 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241507256448, Pid 1126454]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.0.0.129:8092::10.0.0.129:51048 > Passive session Accept complete src/contrail-common/io/tcp_session.cc 256 2025-06-02 Mon 02:13:10:818.905 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241507256448, Pid 1126454]: TCP [SYS_DEBUG]: TcpSessionMessageLog: Session 10.0.0.129:8092::10.0.0.129:51048 < Read failed due to error asio.ssl 335544539 : short read src/contrail-common/io/tcp_session.cc 504 2025-06-02 Mon 02:13:35:532.732 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241344993024, Pid 1126454]: Processing scm::EvSandeshSend in state Idle 2025-06-02 Mon 02:13:35:532.886 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241344993024, Pid 1126454]: SANDESH: Send FAILED: 1748830415532561 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-juju-os-542-1:Control:contrail-dns:0 sm_queue_count = 0 sm_stats= [ ev_stats= [ [ [ event = scm::EvCollectorUpdate enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvConnectTimerExpired enqueues = 10 dequeues = 10 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvIdleHoldTimerExpired enqueues = 9 dequeues = 9 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvSandeshSend enqueues = 87 dequeues = 87 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvStart enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpDeleteSession enqueues = 10 dequeues = 10 enqueue_fails = 0 dequeue_fails = 0 ], [ event = enqueues = 118 dequeues = 118 enqueue_fails = 0 dequeue_fails = 0 ], ] ] state = Idle state_since = 1748830386553233 last_state = Idle last_event = scm::EvSandeshSend last_event_at = 1748830386553434 ] max_sm_queue_count = 5 ] 2025-06-02 Mon 02:13:35:532.904 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241344993024, Pid 1126454]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-06-02 Mon 02:13:35:532.924 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241344993024, Pid 1126454]: Processing scm::EvSandeshSend in state Idle 2025-06-02 Mon 02:13:35:533.193 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241344993024, Pid 1126454]: SANDESH: Send FAILED: 1748830415532687 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-juju-os-542-1:Control:contrail-dns:0 client_info= [ status = Idle successful_connections = 0 pid = 1126454 http_port = 8092 start_time = 1748830000967073 collector_name = collector_ip = 10.0.0.129:8086 collector_list= [ [ (*_iter40) = 10.0.0.129:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ConfigCassInitErrorMessage [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 4 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigClientMgrDebug [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 713 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigClientMgrWarning [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 5 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigurationMalformedReferenceWarningLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 8 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = DnsOperational [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 14 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapLinkOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 454 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapNodeOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 257 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 26 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 46 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 20 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveDnsInfo [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionCreate [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionDelete [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 14 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppCreateConnection [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppSslHandShakeMessage [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppUnconsumedEvent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 1489, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 87, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-06-02 Mon 02:13:35:533.223 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241344993024, Pid 1126454]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-06-02 Mon 02:13:36:553.385 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241298810624, Pid 1126454]: Processing scm::EvIdleHoldTimerExpired in state Idle 2025-06-02 Mon 02:13:36:553.898 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241298810624, Pid 1126454]: Connect : Start Connect timer 10.0.0.129:8086 2025-06-02 Mon 02:13:36:554.099 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241298810624, Pid 1126454]: Processing scm::EvSandeshSend in state Connect 2025-06-02 Mon 02:13:36:554.230 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241298810624, Pid 1126454]: SANDESH: Send FAILED: 1748830416553487 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-juju-os-542-1 process_status= [ [ [ module_id = contrail-dns instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.129:8086, ] ] status = Initializing description = Idle : EvIdleHoldTimerExpired -> Connect ], [ type = Database name = Cassandra server_addrs= [ [ (*_iter6) = 10.0.0.129:9041, ] ] status = Up description = Established Cassandra connection ], [ type = Database name = RabbitMQ server_addrs= [ [ (*_iter6) = 10.0.0.129:5673, ] ] status = Up description = RabbitMQ connection established ], ] ] flag_infos= [ [ ] ] description = Collector connection down ], ] ] ] 2025-06-02 Mon 02:13:36:554.249 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241298810624, Pid 1126454]: Wrong state: Connect for event: EvSandeshSend message: NodeStatusUVE 2025-06-02 Mon 02:13:36:554.264 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241298810624, Pid 1126454]: Processing scm::EvSandeshSend in state Connect 2025-06-02 Mon 02:13:36:555.012 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241298810624, Pid 1126454]: SANDESH: Send FAILED: 1748830416554071 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-juju-os-542-1:Control:contrail-dns:0 client_info= [ status = Connect successful_connections = 0 pid = 1126454 http_port = 8092 start_time = 1748830000967073 collector_name = collector_ip = 10.0.0.129:8086 collector_list= [ [ (*_iter40) = 10.0.0.129:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ConfigCassInitErrorMessage [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 4 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigClientMgrDebug [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 713 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigClientMgrWarning [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 5 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigurationMalformedReferenceWarningLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 8 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = DnsOperational [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 14 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapLinkOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 454 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapNodeOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 257 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 26 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 48 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 20 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveDnsInfo [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionCreate [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionDelete [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 14 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppCreateConnection [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppSslHandShakeMessage [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppUnconsumedEvent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 1489, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 89, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-06-02 Mon 02:13:36:555.063 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241298810624, Pid 1126454]: Wrong state: Connect for event: EvSandeshSend message: SandeshModuleClientTrace 2025-06-02 Mon 02:14:05:533.227 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241344993024, Pid 1126454]: Processing scm::EvSandeshSend in state Connect 2025-06-02 Mon 02:14:05:533.377 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241344993024, Pid 1126454]: SANDESH: Send FAILED: 1748830445532870 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-juju-os-542-1:Control:contrail-dns:0 sm_queue_count = 0 sm_stats= [ ev_stats= [ [ [ event = scm::EvCollectorUpdate enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvConnectTimerExpired enqueues = 10 dequeues = 10 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvIdleHoldTimerExpired enqueues = 10 dequeues = 10 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvSandeshSend enqueues = 91 dequeues = 91 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvStart enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpDeleteSession enqueues = 10 dequeues = 10 enqueue_fails = 0 dequeue_fails = 0 ], [ event = enqueues = 123 dequeues = 123 enqueue_fails = 0 dequeue_fails = 0 ], ] ] state = Connect state_since = 1748830416553496 last_state = Connect last_event = scm::EvSandeshSend last_event_at = 1748830416554262 ] session_stats= [ num_recv_msg = 0 num_recv_msg_fail = 0 num_recv_fail = 0 num_send_msg = 0 num_send_msg_fail = 0 num_send_buffer_fail = 0 num_wait_msgq_enqueue = 0 num_wait_msgq_dequeue = 0 num_write_ready_cb_error = 0 ] session_rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] session_tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] max_sm_queue_count = 5 ] 2025-06-02 Mon 02:14:05:533.414 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241344993024, Pid 1126454]: Wrong state: Connect for event: EvSandeshSend message: SandeshModuleClientTrace 2025-06-02 Mon 02:14:05:533.425 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241344993024, Pid 1126454]: Processing scm::EvSandeshSend in state Connect 2025-06-02 Mon 02:14:05:533.743 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241344993024, Pid 1126454]: SANDESH: Send FAILED: 1748830445533186 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-juju-os-542-1:Control:contrail-dns:0 client_info= [ status = Connect successful_connections = 0 pid = 1126454 http_port = 8092 start_time = 1748830000967073 collector_name = collector_ip = 10.0.0.129:8086 collector_list= [ [ (*_iter40) = 10.0.0.129:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ConfigCassInitErrorMessage [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 4 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigClientMgrDebug [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 713 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigClientMgrWarning [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 5 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigurationMalformedReferenceWarningLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 8 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = DnsOperational [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 14 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapLinkOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 454 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapNodeOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 257 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 27 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 49 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 20 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveDnsInfo [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionCreate [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionDelete [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 14 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppCreateConnection [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppSslHandShakeMessage [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppUnconsumedEvent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 1489, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 91, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-06-02 Mon 02:14:05:534.943 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241344993024, Pid 1126454]: Wrong state: Connect for event: EvSandeshSend message: SandeshModuleClientTrace 2025-06-02 Mon 02:14:06:554.076 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241332397824, Pid 1126454]: 10.0.0.129:8086 EvConnectTimerExpired in state Connect 2025-06-02 Mon 02:14:06:554.257 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241328199424, Pid 1126454]: Processing scm::EvConnectTimerExpired in state Connect 2025-06-02 Mon 02:14:06:554.312 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241328199424, Pid 1126454]: Connect : EvConnectTimerExpired 2025-06-02 Mon 02:14:06:554.406 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241328199424, Pid 1126454]: Idle 2025-06-02 Mon 02:14:06:554.508 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241328199424, Pid 1126454]: Processing scm::EvSandeshSend in state Idle 2025-06-02 Mon 02:14:06:554.537 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241328199424, Pid 1126454]: SANDESH: Send FAILED: 1748830446554348 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-juju-os-542-1 process_status= [ [ [ module_id = contrail-dns instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvConnectTimerExpired ], [ type = Database name = Cassandra server_addrs= [ [ (*_iter6) = 10.0.0.129:9041, ] ] status = Up description = Established Cassandra connection ], [ type = Database name = RabbitMQ server_addrs= [ [ (*_iter6) = 10.0.0.129:5673, ] ] status = Up description = RabbitMQ connection established ], ] ] flag_infos= [ [ ] ] description = Collector connection down ], ] ] ] 2025-06-02 Mon 02:14:06:554.556 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241328199424, Pid 1126454]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-06-02 Mon 02:14:06:554.572 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241328199424, Pid 1126454]: Processing scm::EvTcpDeleteSession in state Idle 2025-06-02 Mon 02:14:06:554.604 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241328199424, Pid 1126454]: Processing scm::EvSandeshSend in state Idle 2025-06-02 Mon 02:14:06:554.957 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241328199424, Pid 1126454]: SANDESH: Send FAILED: 1748830446554494 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-juju-os-542-1:Control:contrail-dns:0 client_info= [ status = Idle successful_connections = 0 pid = 1126454 http_port = 8092 start_time = 1748830000967073 collector_name = collector_ip = 10.0.0.129:8086 collector_list= [ [ (*_iter40) = 10.0.0.129:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ConfigCassInitErrorMessage [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 4 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigClientMgrDebug [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 713 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigClientMgrWarning [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 5 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigurationMalformedReferenceWarningLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 8 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = DnsOperational [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 14 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapLinkOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 454 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapNodeOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 257 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 27 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 51 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 20 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveDnsInfo [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionCreate [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionDelete [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 14 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppCreateConnection [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppSslHandShakeMessage [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppUnconsumedEvent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 1489, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 93, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-06-02 Mon 02:14:06:554.981 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241328199424, Pid 1126454]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-06-02 Mon 02:14:35:533.504 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241315604224, Pid 1126454]: Processing scm::EvSandeshSend in state Idle 2025-06-02 Mon 02:14:35:533.935 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241315604224, Pid 1126454]: SANDESH: Send FAILED: 1748830475533306 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-juju-os-542-1:Control:contrail-dns:0 sm_queue_count = 0 sm_stats= [ ev_stats= [ [ [ event = scm::EvCollectorUpdate enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvConnectTimerExpired enqueues = 11 dequeues = 11 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvIdleHoldTimerExpired enqueues = 10 dequeues = 10 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvSandeshSend enqueues = 95 dequeues = 95 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvStart enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpDeleteSession enqueues = 11 dequeues = 11 enqueue_fails = 0 dequeue_fails = 0 ], [ event = enqueues = 129 dequeues = 129 enqueue_fails = 0 dequeue_fails = 0 ], ] ] state = Idle state_since = 1748830446554404 last_state = Idle last_event = scm::EvSandeshSend last_event_at = 1748830446554602 ] max_sm_queue_count = 5 ] 2025-06-02 Mon 02:14:35:533.959 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241315604224, Pid 1126454]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-06-02 Mon 02:14:35:533.972 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241315604224, Pid 1126454]: Processing scm::EvSandeshSend in state Idle 2025-06-02 Mon 02:14:35:534.492 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241315604224, Pid 1126454]: SANDESH: Send FAILED: 1748830475533460 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-juju-os-542-1:Control:contrail-dns:0 client_info= [ status = Idle successful_connections = 0 pid = 1126454 http_port = 8092 start_time = 1748830000967073 collector_name = collector_ip = 10.0.0.129:8086 collector_list= [ [ (*_iter40) = 10.0.0.129:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ConfigCassInitErrorMessage [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 4 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigClientMgrDebug [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 713 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigClientMgrWarning [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 5 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigurationMalformedReferenceWarningLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 8 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = DnsOperational [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 14 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapLinkOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 454 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapNodeOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 257 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 28 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 52 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 20 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveDnsInfo [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionCreate [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionDelete [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 14 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppCreateConnection [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppSslHandShakeMessage [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppUnconsumedEvent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 1489, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 95, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-06-02 Mon 02:14:35:534.533 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241315604224, Pid 1126454]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-06-02 Mon 02:14:36:554.541 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241336596224, Pid 1126454]: Processing scm::EvIdleHoldTimerExpired in state Idle 2025-06-02 Mon 02:14:36:554.780 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241336596224, Pid 1126454]: Connect : Start Connect timer 10.0.0.129:8086 2025-06-02 Mon 02:14:36:554.909 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241336596224, Pid 1126454]: Processing scm::EvSandeshSend in state Connect 2025-06-02 Mon 02:14:36:554.939 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241336596224, Pid 1126454]: SANDESH: Send FAILED: 1748830476554613 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-juju-os-542-1 process_status= [ [ [ module_id = contrail-dns instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.129:8086, ] ] status = Initializing description = Idle : EvIdleHoldTimerExpired -> Connect ], [ type = Database name = Cassandra server_addrs= [ [ (*_iter6) = 10.0.0.129:9041, ] ] status = Up description = Established Cassandra connection ], [ type = Database name = RabbitMQ server_addrs= [ [ (*_iter6) = 10.0.0.129:5673, ] ] status = Up description = RabbitMQ connection established ], ] ] flag_infos= [ [ ] ] description = Collector connection down ], ] ] ] 2025-06-02 Mon 02:14:36:554.948 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241336596224, Pid 1126454]: Wrong state: Connect for event: EvSandeshSend message: NodeStatusUVE 2025-06-02 Mon 02:14:36:554.956 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241336596224, Pid 1126454]: Processing scm::EvSandeshSend in state Connect 2025-06-02 Mon 02:14:36:555.218 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241336596224, Pid 1126454]: SANDESH: Send FAILED: 1748830476554892 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-juju-os-542-1:Control:contrail-dns:0 client_info= [ status = Connect successful_connections = 0 pid = 1126454 http_port = 8092 start_time = 1748830000967073 collector_name = collector_ip = 10.0.0.129:8086 collector_list= [ [ (*_iter40) = 10.0.0.129:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ConfigCassInitErrorMessage [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 4 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigClientMgrDebug [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 713 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigClientMgrWarning [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 5 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigurationMalformedReferenceWarningLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 8 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = DnsOperational [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 14 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapLinkOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 454 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapNodeOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 257 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 28 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 54 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 20 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveDnsInfo [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionCreate [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionDelete [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 14 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppCreateConnection [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppSslHandShakeMessage [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppUnconsumedEvent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 1489, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 97, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-06-02 Mon 02:14:36:555.853 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241336596224, Pid 1126454]: Wrong state: Connect for event: EvSandeshSend message: SandeshModuleClientTrace 2025-06-02 Mon 02:15:05:533.843 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241311405824, Pid 1126454]: Processing scm::EvSandeshSend in state Connect 2025-06-02 Mon 02:15:05:533.961 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241311405824, Pid 1126454]: SANDESH: Send FAILED: 1748830505533737 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-juju-os-542-1:Control:contrail-dns:0 sm_queue_count = 0 sm_stats= [ ev_stats= [ [ [ event = scm::EvCollectorUpdate enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvConnectTimerExpired enqueues = 11 dequeues = 11 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvIdleHoldTimerExpired enqueues = 11 dequeues = 11 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvSandeshSend enqueues = 99 dequeues = 99 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvStart enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpDeleteSession enqueues = 11 dequeues = 11 enqueue_fails = 0 dequeue_fails = 0 ], [ event = enqueues = 134 dequeues = 134 enqueue_fails = 0 dequeue_fails = 0 ], ] ] state = Connect state_since = 1748830476554627 last_state = Connect last_event = scm::EvSandeshSend last_event_at = 1748830476554954 ] session_stats= [ num_recv_msg = 0 num_recv_msg_fail = 0 num_recv_fail = 0 num_send_msg = 0 num_send_msg_fail = 0 num_send_buffer_fail = 0 num_wait_msgq_enqueue = 0 num_wait_msgq_dequeue = 0 num_write_ready_cb_error = 0 ] session_rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] session_tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] max_sm_queue_count = 5 ] 2025-06-02 Mon 02:15:05:533.986 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241311405824, Pid 1126454]: Wrong state: Connect for event: EvSandeshSend message: SandeshModuleClientTrace 2025-06-02 Mon 02:15:05:533.995 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241311405824, Pid 1126454]: Processing scm::EvSandeshSend in state Connect 2025-06-02 Mon 02:15:05:534.328 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241311405824, Pid 1126454]: SANDESH: Send FAILED: 1748830505533819 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-juju-os-542-1:Control:contrail-dns:0 client_info= [ status = Connect successful_connections = 0 pid = 1126454 http_port = 8092 start_time = 1748830000967073 collector_name = collector_ip = 10.0.0.129:8086 collector_list= [ [ (*_iter40) = 10.0.0.129:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ConfigCassInitErrorMessage [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 4 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigClientMgrDebug [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 713 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigClientMgrWarning [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 5 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigurationMalformedReferenceWarningLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 8 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = DnsOperational [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 14 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapLinkOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 454 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapNodeOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 257 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 29 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 55 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 20 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveDnsInfo [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionCreate [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionDelete [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 14 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppCreateConnection [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppSslHandShakeMessage [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppUnconsumedEvent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 1489, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 99, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-06-02 Mon 02:15:05:534.353 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241311405824, Pid 1126454]: Wrong state: Connect for event: EvSandeshSend message: SandeshModuleClientTrace 2025-06-02 Mon 02:15:06:554.968 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241349191424, Pid 1126454]: 10.0.0.129:8086 EvConnectTimerExpired in state Connect 2025-06-02 Mon 02:15:06:555.059 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241332397824, Pid 1126454]: Processing scm::EvConnectTimerExpired in state Connect 2025-06-02 Mon 02:15:06:555.170 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241332397824, Pid 1126454]: Connect : EvConnectTimerExpired 2025-06-02 Mon 02:15:06:555.268 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241332397824, Pid 1126454]: Idle 2025-06-02 Mon 02:15:06:555.383 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241332397824, Pid 1126454]: Processing scm::EvSandeshSend in state Idle 2025-06-02 Mon 02:15:06:555.418 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241332397824, Pid 1126454]: SANDESH: Send FAILED: 1748830506555210 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-juju-os-542-1 process_status= [ [ [ module_id = contrail-dns instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvConnectTimerExpired ], [ type = Database name = Cassandra server_addrs= [ [ (*_iter6) = 10.0.0.129:9041, ] ] status = Up description = Established Cassandra connection ], [ type = Database name = RabbitMQ server_addrs= [ [ (*_iter6) = 10.0.0.129:5673, ] ] status = Up description = RabbitMQ connection established ], ] ] flag_infos= [ [ ] ] description = Collector connection down ], ] ] ] 2025-06-02 Mon 02:15:06:555.442 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241332397824, Pid 1126454]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-06-02 Mon 02:15:06:555.456 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241332397824, Pid 1126454]: Processing scm::EvTcpDeleteSession in state Idle 2025-06-02 Mon 02:15:06:555.499 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241332397824, Pid 1126454]: Processing scm::EvSandeshSend in state Idle 2025-06-02 Mon 02:15:06:555.831 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241332397824, Pid 1126454]: SANDESH: Send FAILED: 1748830506555368 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-juju-os-542-1:Control:contrail-dns:0 client_info= [ status = Idle successful_connections = 0 pid = 1126454 http_port = 8092 start_time = 1748830000967073 collector_name = collector_ip = 10.0.0.129:8086 collector_list= [ [ (*_iter40) = 10.0.0.129:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ConfigCassInitErrorMessage [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 4 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigClientMgrDebug [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 713 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigClientMgrWarning [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 5 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigurationMalformedReferenceWarningLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 8 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = DnsOperational [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 14 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapLinkOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 454 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapNodeOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 257 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 29 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 57 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 20 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveDnsInfo [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionCreate [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionDelete [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 14 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppCreateConnection [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppSslHandShakeMessage [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppUnconsumedEvent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 1489, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 101, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-06-02 Mon 02:15:06:555.864 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241332397824, Pid 1126454]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-06-02 Mon 02:15:35:534.096 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241349191424, Pid 1126454]: Processing scm::EvSandeshSend in state Idle 2025-06-02 Mon 02:15:35:534.191 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241349191424, Pid 1126454]: SANDESH: Send FAILED: 1748830535533959 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-juju-os-542-1:Control:contrail-dns:0 sm_queue_count = 0 sm_stats= [ ev_stats= [ [ [ event = scm::EvCollectorUpdate enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvConnectTimerExpired enqueues = 12 dequeues = 12 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvIdleHoldTimerExpired enqueues = 11 dequeues = 11 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvSandeshSend enqueues = 103 dequeues = 103 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvStart enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpDeleteSession enqueues = 12 dequeues = 12 enqueue_fails = 0 dequeue_fails = 0 ], [ event = enqueues = 140 dequeues = 140 enqueue_fails = 0 dequeue_fails = 0 ], ] ] state = Idle state_since = 1748830506555266 last_state = Idle last_event = scm::EvSandeshSend last_event_at = 1748830506555497 ] max_sm_queue_count = 5 ] 2025-06-02 Mon 02:15:35:534.205 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241349191424, Pid 1126454]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-06-02 Mon 02:15:35:534.213 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241349191424, Pid 1126454]: Processing scm::EvSandeshSend in state Idle 2025-06-02 Mon 02:15:35:534.498 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241349191424, Pid 1126454]: SANDESH: Send FAILED: 1748830535534071 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-juju-os-542-1:Control:contrail-dns:0 client_info= [ status = Idle successful_connections = 0 pid = 1126454 http_port = 8092 start_time = 1748830000967073 collector_name = collector_ip = 10.0.0.129:8086 collector_list= [ [ (*_iter40) = 10.0.0.129:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ConfigCassInitErrorMessage [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 4 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigClientMgrDebug [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 713 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigClientMgrWarning [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 5 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigurationMalformedReferenceWarningLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 8 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = DnsOperational [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 14 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapLinkOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 454 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapNodeOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 257 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 30 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 58 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 20 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveDnsInfo [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionCreate [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionDelete [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 14 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppCreateConnection [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppSslHandShakeMessage [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppUnconsumedEvent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 1489, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 103, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-06-02 Mon 02:15:35:534.527 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241349191424, Pid 1126454]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-06-02 Mon 02:15:36:555.411 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241315604224, Pid 1126454]: Processing scm::EvIdleHoldTimerExpired in state Idle 2025-06-02 Mon 02:15:36:555.646 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241315604224, Pid 1126454]: Connect : Start Connect timer 10.0.0.129:8086 2025-06-02 Mon 02:15:36:555.764 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241315604224, Pid 1126454]: Processing scm::EvSandeshSend in state Connect 2025-06-02 Mon 02:15:36:555.794 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241315604224, Pid 1126454]: SANDESH: Send FAILED: 1748830536555490 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-juju-os-542-1 process_status= [ [ [ module_id = contrail-dns instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 10.0.0.129:8086, ] ] status = Initializing description = Idle : EvIdleHoldTimerExpired -> Connect ], [ type = Database name = Cassandra server_addrs= [ [ (*_iter6) = 10.0.0.129:9041, ] ] status = Up description = Established Cassandra connection ], [ type = Database name = RabbitMQ server_addrs= [ [ (*_iter6) = 10.0.0.129:5673, ] ] status = Up description = RabbitMQ connection established ], ] ] flag_infos= [ [ ] ] description = Collector connection down ], ] ] ] 2025-06-02 Mon 02:15:36:555.803 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241315604224, Pid 1126454]: Wrong state: Connect for event: EvSandeshSend message: NodeStatusUVE 2025-06-02 Mon 02:15:36:555.811 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241315604224, Pid 1126454]: Processing scm::EvSandeshSend in state Connect 2025-06-02 Mon 02:15:36:556.114 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241315604224, Pid 1126454]: SANDESH: Send FAILED: 1748830536555744 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-juju-os-542-1:Control:contrail-dns:0 client_info= [ status = Connect successful_connections = 0 pid = 1126454 http_port = 8092 start_time = 1748830000967073 collector_name = collector_ip = 10.0.0.129:8086 collector_list= [ [ (*_iter40) = 10.0.0.129:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ConfigCassInitErrorMessage [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 4 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigClientMgrDebug [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 713 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigClientMgrWarning [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 5 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigurationMalformedReferenceWarningLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 8 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = DnsOperational [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 14 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapLinkOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 454 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapNodeOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 257 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 30 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 60 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 20 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveDnsInfo [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionCreate [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionDelete [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 14 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppCreateConnection [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppSslHandShakeMessage [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppUnconsumedEvent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 1489, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 105, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-06-02 Mon 02:15:36:556.146 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241315604224, Pid 1126454]: Wrong state: Connect for event: EvSandeshSend message: SandeshModuleClientTrace 2025-06-02 Mon 02:16:05:534.428 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241307207424, Pid 1126454]: Processing scm::EvSandeshSend in state Connect 2025-06-02 Mon 02:16:05:534.702 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241307207424, Pid 1126454]: SANDESH: Send FAILED: 1748830565534231 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-juju-os-542-1:Control:contrail-dns:0 sm_queue_count = 0 sm_stats= [ ev_stats= [ [ [ event = scm::EvCollectorUpdate enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvConnectTimerExpired enqueues = 12 dequeues = 12 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvIdleHoldTimerExpired enqueues = 12 dequeues = 12 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvSandeshSend enqueues = 107 dequeues = 107 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvStart enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpDeleteSession enqueues = 12 dequeues = 12 enqueue_fails = 0 dequeue_fails = 0 ], [ event = enqueues = 145 dequeues = 145 enqueue_fails = 0 dequeue_fails = 0 ], ] ] state = Connect state_since = 1748830536555497 last_state = Connect last_event = scm::EvSandeshSend last_event_at = 1748830536555810 ] session_stats= [ num_recv_msg = 0 num_recv_msg_fail = 0 num_recv_fail = 0 num_send_msg = 0 num_send_msg_fail = 0 num_send_buffer_fail = 0 num_wait_msgq_enqueue = 0 num_wait_msgq_dequeue = 0 num_write_ready_cb_error = 0 ] session_rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] session_tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] max_sm_queue_count = 5 ] 2025-06-02 Mon 02:16:05:534.766 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241307207424, Pid 1126454]: Wrong state: Connect for event: EvSandeshSend message: SandeshModuleClientTrace 2025-06-02 Mon 02:16:05:534.786 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241307207424, Pid 1126454]: Processing scm::EvSandeshSend in state Connect 2025-06-02 Mon 02:16:05:535.298 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241307207424, Pid 1126454]: SANDESH: Send FAILED: 1748830565534332 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-juju-os-542-1:Control:contrail-dns:0 client_info= [ status = Connect successful_connections = 0 pid = 1126454 http_port = 8092 start_time = 1748830000967073 collector_name = collector_ip = 10.0.0.129:8086 collector_list= [ [ (*_iter40) = 10.0.0.129:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ConfigCassInitErrorMessage [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 4 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigClientMgrDebug [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 713 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigClientMgrWarning [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 5 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigurationMalformedReferenceWarningLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 8 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = DnsOperational [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 14 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapLinkOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 454 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapNodeOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 257 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 31 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 61 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 20 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveDnsInfo [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionCreate [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionDelete [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 14 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppCreateConnection [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppSslHandShakeMessage [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppUnconsumedEvent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 1489, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 107, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-06-02 Mon 02:16:05:535.343 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241307207424, Pid 1126454]: Wrong state: Connect for event: EvSandeshSend message: SandeshModuleClientTrace 2025-06-02 Mon 02:16:06:555.791 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241315604224, Pid 1126454]: 10.0.0.129:8086 EvConnectTimerExpired in state Connect 2025-06-02 Mon 02:16:06:555.882 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241303009024, Pid 1126454]: Processing scm::EvConnectTimerExpired in state Connect 2025-06-02 Mon 02:16:06:555.965 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241303009024, Pid 1126454]: Connect : EvConnectTimerExpired 2025-06-02 Mon 02:16:06:556.057 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241303009024, Pid 1126454]: Idle 2025-06-02 Mon 02:16:06:556.169 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241303009024, Pid 1126454]: Processing scm::EvSandeshSend in state Idle 2025-06-02 Mon 02:16:06:556.198 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241303009024, Pid 1126454]: SANDESH: Send FAILED: 1748830566556000 [SYS_NOTICE]: NodeStatusUVE: data= [ name = cn-jenkins-deploy-platform-juju-os-542-1 process_status= [ [ [ module_id = contrail-dns instance_id = 0 state = Non-Functional connection_infos= [ [ [ type = Collector name = server_addrs= [ [ (*_iter6) = 0.0.0.0:0, ] ] status = Down description = Connect : EvConnectTimerExpired ], [ type = Database name = Cassandra server_addrs= [ [ (*_iter6) = 10.0.0.129:9041, ] ] status = Up description = Established Cassandra connection ], [ type = Database name = RabbitMQ server_addrs= [ [ (*_iter6) = 10.0.0.129:5673, ] ] status = Up description = RabbitMQ connection established ], ] ] flag_infos= [ [ ] ] description = Collector connection down ], ] ] ] 2025-06-02 Mon 02:16:06:556.217 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241303009024, Pid 1126454]: Wrong state: Idle for event: EvSandeshSend message: NodeStatusUVE 2025-06-02 Mon 02:16:06:556.226 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241303009024, Pid 1126454]: Processing scm::EvTcpDeleteSession in state Idle 2025-06-02 Mon 02:16:06:556.258 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241303009024, Pid 1126454]: Processing scm::EvSandeshSend in state Idle 2025-06-02 Mon 02:16:06:556.541 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241303009024, Pid 1126454]: SANDESH: Send FAILED: 1748830566556154 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-juju-os-542-1:Control:contrail-dns:0 client_info= [ status = Idle successful_connections = 0 pid = 1126454 http_port = 8092 start_time = 1748830000967073 collector_name = collector_ip = 10.0.0.129:8086 collector_list= [ [ (*_iter40) = 10.0.0.129:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ConfigCassInitErrorMessage [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 4 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigClientMgrDebug [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 713 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigClientMgrWarning [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 5 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigurationMalformedReferenceWarningLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 8 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = DnsOperational [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 14 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapLinkOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 454 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapNodeOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 257 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 31 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 63 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 20 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveDnsInfo [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionCreate [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionDelete [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 14 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppCreateConnection [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppSslHandShakeMessage [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppUnconsumedEvent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 1489, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 109, _iter59->first = sent _iter59->second = 0, ] ] ] 2025-06-02 Mon 02:16:06:556.565 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241303009024, Pid 1126454]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-06-02 Mon 02:16:35:535.104 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241307207424, Pid 1126454]: Processing scm::EvSandeshSend in state Idle 2025-06-02 Mon 02:16:35:535.228 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241307207424, Pid 1126454]: SANDESH: Send FAILED: 1748830595534554 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-juju-os-542-1:Control:contrail-dns:0 sm_queue_count = 0 sm_stats= [ ev_stats= [ [ [ event = scm::EvCollectorUpdate enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvConnectTimerExpired enqueues = 13 dequeues = 13 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvIdleHoldTimerExpired enqueues = 12 dequeues = 12 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvSandeshSend enqueues = 111 dequeues = 111 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvStart enqueues = 1 dequeues = 1 enqueue_fails = 0 dequeue_fails = 0 ], [ event = scm::EvTcpDeleteSession enqueues = 13 dequeues = 13 enqueue_fails = 0 dequeue_fails = 0 ], [ event = enqueues = 151 dequeues = 151 enqueue_fails = 0 dequeue_fails = 0 ], ] ] state = Idle state_since = 1748830566556055 last_state = Idle last_event = scm::EvSandeshSend last_event_at = 1748830566556256 ] max_sm_queue_count = 5 ] 2025-06-02 Mon 02:16:35:535.250 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241307207424, Pid 1126454]: Wrong state: Idle for event: EvSandeshSend message: SandeshModuleClientTrace 2025-06-02 Mon 02:16:35:535.268 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241307207424, Pid 1126454]: Processing scm::EvSandeshSend in state Idle 2025-06-02 Mon 02:16:35:544.060 UTC cn-jenkins-deploy-platform-juju-os-542-1 [Thread 140241307207424, Pid 1126454]: SANDESH: Send FAILED: 1748830595534721 [SYS_NOTICE]: SandeshModuleClientTrace: data= [ name = cn-jenkins-deploy-platform-juju-os-542-1:Control:contrail-dns:0 client_info= [ status = Idle successful_connections = 0 pid = 1126454 http_port = 8092 start_time = 1748830000967073 collector_name = collector_ip = 10.0.0.129:8086 collector_list= [ [ (*_iter40) = 10.0.0.129:8086, ] ] rx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] tx_socket_stats= [ bytes = 0 calls = 0 average_bytes = 0 blocked_duration = 00:00:00 blocked_count = 0 average_blocked_duration = errors = 0 ] ] msg_type_agg= [ [ _iter58->first = ConfigCassInitErrorMessage [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 4 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigClientMgrDebug [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 713 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigClientMgrWarning [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 5 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = ConfigurationMalformedReferenceWarningLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 8 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = DnsOperational [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 14 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapLinkOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 454 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = IFMapNodeOperation [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 257 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = NodeStatusUVE [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 32 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = SandeshModuleClientTrace [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 1 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 64 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpServerMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = TcpSessionMessageLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 20 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = UveDnsInfo [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 1 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionCreate [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionDelete [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 1 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppConnectionLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 14 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 0 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppCreateConnection [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppEventLog [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 3 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppSslHandShakeMessage [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], _iter58->first = XmppUnconsumedEvent [ messages_sent = 0 messages_sent_dropped_no_queue = 0 messages_sent_dropped_no_client = 0 messages_sent_dropped_no_session = 0 messages_sent_dropped_queue_level = 0 messages_sent_dropped_client_send_failed = 0 messages_sent_dropped_session_not_connected = 0 messages_sent_dropped_header_write_failed = 0 messages_sent_dropped_write_failed = 0 messages_sent_dropped_wrong_client_sm_state = 0 messages_sent_dropped_validation_failed = 0 messages_sent_dropped_rate_limited = 0 messages_sent_dropped_sending_disabled = 2 messages_sent_dropped_sending_to_syslog = 0 ], ] ] tx_msg_agg= [ [ _iter59->first = dropped_client_send_failed _iter59->second = 0, _iter59->first = dropped_header_write_failed _iter59->second = 0, _iter59->first = dropped_no_client _iter59->second = 1, _iter59->first = dropped_no_queue _iter59->second = 0, _iter59->first = dropped_no_session _iter59->second = 0, _iter59->first = dropped_queue_level _iter59->second = 0, _iter59->first = dropped_rate_limited _iter59->second = 0, _iter59->first = dropped_sending_disabled _iter59->second = 1489, _iter59->first = dropped_sending_to_syslog _iter59->second = 0, _iter59->first = dropped_session_not_connected _iter59->second = 0, _iter59->first = dropped_validation_failed _iter59->second = 0, _iter59->first = dropped_write_failed _iter59->second = 0, _iter59->first = dropped_wrong_client_sm_state _iter59->second = 111, _iter59->first = sent _iter59->second = 0, ] ] ]