2025-10-13 01:49:16,713 - INFO - Using existing project ['admin_domain', 'admin'](5fc4bc8d-4b10-4e41-99a1-5d01acfe2c8e)
2025-10-13 01:49:16,720 - INFO - ================================================================================
2025-10-13 01:49:16,720 - INFO - STARTING TEST    : test_db_manage
2025-10-13 01:49:16,720 - INFO - TEST DESCRIPTION :  Test db_manage.py tool
        
2025-10-13 01:49:17,009 - DEBUG - Nothing to compare xmpp stats {'10.0.0.129': {'10.0.0.129': '0'}} with
2025-10-13 01:49:17,009 - INFO - Initial checks done. Running the testcase now
2025-10-13 01:49:17,009 - INFO - 
2025-10-13 01:49:19,547 - DEBUG - 2025-10-13 01:49:19,370 INFO: (v1.41) Checker check_zk_mode_and_node_count: Success
2025-10-13 01:49:19,419 ERROR: (v1.41) Checker check_cassandra_keyspace_replication: Exception, NoHostAvailable
Python 3.9.21: /usr/bin/python3
Mon Oct 13 01:49:19 2025
A problem occurred in a Python script.  Here is the sequence of
function calls leading up to the error, in the order they occurred.
 /usr/local/lib/python3.9/site-packages/vnc_cfg_api_server/db_manage.py in wrapper(*args=(,), **kwargs={})
 1535             self = args[0]
 1536             try:
 1537                 errors = func(*args, **kwargs)
 1538                 if not errors:
 1539                     self._logger.info('(v%s) Checker %s: Success', __version__,
errors undefined
func = 
args = (,)
kwargs = {}
 /usr/local/lib/python3.9/site-packages/vnc_cfg_api_server/db_manage.py in check_cassandra_keyspace_replication(self=)
 1635                 raise CassandraConnectionError("error, {}: {}".format(
 1636                     error, utils.detailed_traceback()))
 1637             temp_cluster.connect()
 1638             for ks_name, _ in self._db_info:
 1639                 if self._api_args.cluster_id:
temp_cluster = 
temp_cluster.connect = >
 /cassandra/cluster.py in cassandra.cluster.Cluster.connect()
 /cassandra/cluster.py in cassandra.cluster.Cluster.connect()
 /cassandra/cluster.py in cassandra.cluster.Cluster.connect()
 /cassandra/cluster.py in cassandra.cluster.ControlConnection.connect()
 /cassandra/cluster.py in cassandra.cluster.ControlConnection._reconnect_internal()
NoHostAvailable: ('Unable to connect to any servers', {'10.0.0.129:9041': ConnectionResetError(104, 'Connection reset by peer')})
    __cause__ = None
    __class__ = 
    __context__ = None
    __delattr__ = 
    __dict__ = {'errors': {'10.0.0.129:9041': ConnectionResetError(104, 'Connection reset by peer')}}
    __dir__ = 
    __doc__ = '\n    Raised when an operation is attempted but a...ct, closed, or resulted in errors when used.\n    '
    __eq__ = 
    __format__ = 
    __ge__ = 
    __getattribute__ = 
    __gt__ = 
    __hash__ = 
    __init__ = 
    __init_subclass__ = 
    __le__ = 
    __lt__ = 
    __module__ = 'cassandra.cluster'
    __ne__ = 
    __new__ = 
    __reduce__ = 
    __reduce_ex__ = 
    __repr__ = 
    __setattr__ = 
    __setstate__ = 
    __sizeof__ = 
    __str__ = 
    __subclasshook__ = 
    __suppress_context__ = False
    __traceback__ = 
    __weakref__ = None
    args = ('Unable to connect to any servers', {'10.0.0.129:9041': ConnectionResetError(104, 'Connection reset by peer')})
    errors = {'10.0.0.129:9041': ConnectionResetError(104, 'Connection reset by peer')}
    with_traceback = 
The above is a description of an error in a Python program.  Here is
the original traceback:
Traceback (most recent call last):
  File "/usr/local/lib/python3.9/site-packages/vnc_cfg_api_server/db_manage.py", line 1537, in wrapper
    errors = func(*args, **kwargs)
  File "/usr/local/lib/python3.9/site-packages/vnc_cfg_api_server/db_manage.py", line 1637, in check_cassandra_keyspace_replication
    temp_cluster.connect()
  File "cassandra/cluster.py", line 1717, in cassandra.cluster.Cluster.connect
  File "cassandra/cluster.py", line 1753, in cassandra.cluster.Cluster.connect
  File "cassandra/cluster.py", line 1740, in cassandra.cluster.Cluster.connect
  File "cassandra/cluster.py", line 3543, in cassandra.cluster.ControlConnection.connect
  File "cassandra/cluster.py", line 3588, in cassandra.cluster.ControlConnection._reconnect_internal
cassandra.cluster.NoHostAvailable: ('Unable to connect to any servers', {'10.0.0.129:9041': ConnectionResetError(104, 'Connection reset by peer')})
Traceback (most recent call last):
  File "/usr/local/lib/python3.9/site-packages/vnc_cfg_api_server/db_manage.py", line 1537, in wrapper
    errors = func(*args, **kwargs)
  File "/usr/local/lib/python3.9/site-packages/vnc_cfg_api_server/db_manage.py", line 1637, in check_cassandra_keyspace_replication
    temp_cluster.connect()
  File "cassandra/cluster.py", line 1717, in cassandra.cluster.Cluster.connect
  File "cassandra/cluster.py", line 1753, in cassandra.cluster.Cluster.connect
  File "cassandra/cluster.py", line 1740, in cassandra.cluster.Cluster.connect
  File "cassandra/cluster.py", line 3543, in cassandra.cluster.ControlConnection.connect
  File "cassandra/cluster.py", line 3588, in cassandra.cluster.ControlConnection._reconnect_internal
cassandra.cluster.NoHostAvailable: ('Unable to connect to any servers', {'10.0.0.129:9041': ConnectionResetError(104, 'Connection reset by peer')})
NoHostAvailable
Python 3.9.21: /usr/bin/python3
Mon Oct 13 01:49:19 2025
A problem occurred in a Python script.  Here is the sequence of
function calls leading up to the error, in the order they occurred.
 /usr/local/bin/db-manage in 
    3 from vnc_cfg_api_server.db_manage import main
    4 if __name__ == '__main__':
    5     if sys.argv[0].endswith('.exe'):
    6         sys.argv[0] = sys.argv[0][:-4]
    7     sys.exit(main())
sys = 
sys.exit = 
main = 
 /usr/local/lib/python3.9/site-packages/vnc_cfg_api_server/db_manage.py in main()
 3451     verb = args.operation
 3452     if 'db_%s' % (verb) in globals():
 3453         return globals()['db_%s' % (verb)](args, api_args)
 3454 
 3455     if getattr(DatabaseChecker, verb, None):
builtinglobals = 
verb = 'check'
args = Namespace(operation='check', api_conf='/etc/cont...1024, log_file='/var/log/contrail/db_manage.log')
api_args = Namespace(conf_file=['/etc/contrail/contrail-api...esh_base.SandeshConfig object at 0x7f5e6e3650a0>)
 /usr/local/lib/python3.9/site-packages/vnc_cfg_api_server/db_manage.py in db_check(args=Namespace(operation='check', api_conf='/etc/cont...1024, log_file='/var/log/contrail/db_manage.log'), api_args=Namespace(conf_file=['/etc/contrail/contrail-api...esh_base.SandeshConfig object at 0x7f5e6e3650a0>))
 3358     # Mode and node count check across all nodes
 3359     db_checker.check_zk_mode_and_node_count()
 3360     db_checker.check_cassandra_keyspace_replication()
 3361     # Obj UUID cassandra inconsistencies
 3362     db_checker.check_obj_mandatory_fields()
db_checker = 
db_checker.check_cassandra_keyspace_replication = >
 /usr/local/lib/python3.9/site-packages/vnc_cfg_api_server/db_manage.py in wrapper(*args=(,), **kwargs={})
 1535             self = args[0]
 1536             try:
 1537                 errors = func(*args, **kwargs)
 1538                 if not errors:
 1539                     self._logger.info('(v%s) Checker %s: Success', __version__,
errors undefined
func = 
args = (,)
kwargs = {}
 /usr/local/lib/python3.9/site-packages/vnc_cfg_api_server/db_manage.py in check_cassandra_keyspace_replication(self=)
 1635                 raise CassandraConnectionError("error, {}: {}".format(
 1636                     error, utils.detailed_traceback()))
 1637             temp_cluster.connect()
 1638             for ks_name, _ in self._db_info:
 1639                 if self._api_args.cluster_id:
temp_cluster = 
temp_cluster.connect = >
 /cassandra/cluster.py in cassandra.cluster.Cluster.connect()
 /cassandra/cluster.py in cassandra.cluster.Cluster.connect()
 /cassandra/cluster.py in cassandra.cluster.Cluster.connect()
 /cassandra/cluster.py in cassandra.cluster.ControlConnection.connect()
 /cassandra/cluster.py in cassandra.cluster.ControlConnection._reconnect_internal()
NoHostAvailable: ('Unable to connect to any servers', {'10.0.0.129:9041': ConnectionResetError(104, 'Connection reset by peer')})
    __cause__ = None
    __class__ = 
    __context__ = None
    __delattr__ = 
    __dict__ = {'errors': {'10.0.0.129:9041': ConnectionResetError(104, 'Connection reset by peer')}}
    __dir__ = 
    __doc__ = '\n    Raised when an operation is attempted but a...ct, closed, or resulted in errors when used.\n    '
    __eq__ = 
    __format__ = 
    __ge__ = 
    __getattribute__ = 
    __gt__ = 
    __hash__ = 
    __init__ = 
    __init_subclass__ = 
    __le__ = 
    __lt__ = 
    __module__ = 'cassandra.cluster'
    __ne__ = 
    __new__ = 
    __reduce__ = 
    __reduce_ex__ = 
    __repr__ = 
    __setattr__ = 
    __setstate__ = 
    __sizeof__ = 
    __str__ = 
    __subclasshook__ = 
    __suppress_context__ = False
    __traceback__ = 
    __weakref__ = None
    args = ('Unable to connect to any servers', {'10.0.0.129:9041': ConnectionResetError(104, 'Connection reset by peer')})
    errors = {'10.0.0.129:9041': ConnectionResetError(104, 'Connection reset by peer')}
    with_traceback = 
The above is a description of an error in a Python program.  Here is
the original traceback:
Traceback (most recent call last):
  File "/usr/local/bin/db-manage", line 7, in 
    sys.exit(main())
  File "/usr/local/lib/python3.9/site-packages/vnc_cfg_api_server/db_manage.py", line 3453, in main
    return globals()['db_%s' % (verb)](args, api_args)
  File "/usr/local/lib/python3.9/site-packages/vnc_cfg_api_server/db_manage.py", line 3360, in db_check
    db_checker.check_cassandra_keyspace_replication()
  File "/usr/local/lib/python3.9/site-packages/vnc_cfg_api_server/db_manage.py", line 1537, in wrapper
    errors = func(*args, **kwargs)
  File "/usr/local/lib/python3.9/site-packages/vnc_cfg_api_server/db_manage.py", line 1637, in check_cassandra_keyspace_replication
    temp_cluster.connect()
  File "cassandra/cluster.py", line 1717, in cassandra.cluster.Cluster.connect
  File "cassandra/cluster.py", line 1753, in cassandra.cluster.Cluster.connect
  File "cassandra/cluster.py", line 1740, in cassandra.cluster.Cluster.connect
  File "cassandra/cluster.py", line 3543, in cassandra.cluster.ControlConnection.connect
  File "cassandra/cluster.py", line 3588, in cassandra.cluster.ControlConnection._reconnect_internal
cassandra.cluster.NoHostAvailable: ('Unable to connect to any servers', {'10.0.0.129:9041': ConnectionResetError(104, 'Connection reset by peer')})
2025-10-13 01:49:19,547 - WARNING - Errors are still present: 2025-10-13 01:49:19,419 ERROR: (v1.41) Checker check_cassandra_keyspace_replication: Exception, NoHostAvailable
2025-10-13 01:49:19,845 - DEBUG - No XMPP flaps were noticed during the test
2025-10-13 01:49:19,845 - INFO - 
2025-10-13 01:49:19,845 - INFO - END TEST : test_db_manage : FAILED[0:00:03]
2025-10-13 01:49:19,845 - INFO - --------------------------------------------------------------------------------