]) created on node: (an-jenkins-deploy-platform-ansible-os-3704-1), Zone: (nova:an-jenkins-deploy-platform-ansible-os-3704-1)
2025-07-23 15:15:47,517 - INFO - ================================================================================
2025-07-23 15:15:47,517 - INFO - STARTING TEST : test_svc_fate_sharing_basic
2025-07-23 15:15:48,871 - DEBUG - Nothing to compare xmpp stats {'10.0.0.50': {'10.20.0.254': '0', '10.20.0.17': '0'}, '10.0.0.130': {'10.20.0.17': '0', '10.20.0.254': '0'}} with
2025-07-23 15:15:48,871 - INFO - Initial checks done. Running the testcase now
2025-07-23 15:15:48,871 - INFO -
2025-07-23 15:15:48,871 - INFO - ================================================================================
2025-07-23 15:15:48,871 - INFO - STARTING TEST : test_svc_fate_sharing_basic
2025-07-23 15:15:48,871 - INFO - TEST DESCRIPTION :
Description: Basic test to validate fate sharing in a multi inline service chain with 2 SIs, 1 SVM each.
Test steps:
1.Creating vm's - vm1 and vm2 in networks vn1 and vn2.
2.Creating 2 inline service instances SI1 and SI2.
3.Creating a service chain by applying the 2 service instances in a policy between t
he VNs.
4.Associate an HC instance with right intf of one of the SVMs
5.Bring down the intf in the step 4
6.Verify that all the re-origined routes(ServiceChain) are deleted from all the RIs of all the SIs
7.Verify the step 6 in both control node and agent
8.Bring up the intf in the step 5
8.Verify that the routes should be re-originated again, and the traffic also starts flowing
Pass criteria: Ping/Route deletion/Route addition should be successful.
Maintainer : ankitja@juniper.net
2025-07-23 15:15:50,166 - DEBUG - Nothing to compare xmpp stats {'10.0.0.50': {'10.20.0.254': '0', '10.20.0.17': '0'}, '10.0.0.130': {'10.20.0.17': '0', '10.20.0.254': '0'}} with
2025-07-23 15:15:50,167 - INFO - Initial checks done. Running the testcase now
2025-07-23 15:15:50,167 - INFO -
2025-07-23 15:15:50,167 - DEBUG - Hosts: {'nova': ['an-jenkins-deploy-platform-ansible-os-3704-2', 'an-jenkins-deploy-platform-ansible-os-3704-1']}
2025-07-23 15:15:50,167 - DEBUG - Creating service template: ['default-domain', 'ctest-service_template_1-31112897']
2025-07-23 15:15:50,488 - DEBUG - Service template: ['default-domain', 'ctest-service_template_1-31112897'] created succesfully
2025-07-23 15:17:31,112 - DEBUG - Installing image tiny_in_net
2025-07-23 15:17:31,112 - DEBUG - Image info {'name': 'tinycore-in-network.qcow2.gz', 'webserver': '', 'location': '/images/tinycore/', 'username': 'tc', 'password': 'secret', 'type': 'os', 'flavor': 'contrail_flavor_tiny', 'params': '--container-format bare --disk-format vmdk --property vmware_disktype="sparse" --property vmware_adaptertype="ide" --property hypervisor_type=qemu', 'name_docker': 'phusion-baseimage-enablesshd', 'vmdk': 'service_chain_images/nat_service/tiny_in_net/tinycore-in-network-flat.vmdk', 'shrinked_vmdk': "'no'"}
2025-07-23 15:17:31,112 - DEBUG - Image build path file:///contrail-test/images/tinycore-in-network.qcow2.gz
2025-07-23 15:17:31,112 - DEBUG - Download image from local file file:///contrail-test/images/tinycore-in-network.qcow2.gz
2025-07-23 15:17:31,112 - DEBUG - Unzip image
2025-07-23 15:17:31,112 - DEBUG - Image download (local=True) (proxy=None): Execute cmd: gunzip -f /contrail-test/images/tinycore-in-network.qcow2.gz
2025-07-23 15:17:31,128 - DEBUG - Try to create image in user context
2025-07-23 15:17:31,207 - DEBUG - Deleting service template: ['default-domain', 'ctest-service_template_1-31112897']
2025-07-23 15:17:31,311 - INFO - Service template: ['default-domain', 'ctest-service_template_1-31112897'] deleted successfully.
2025-07-23 15:17:32,641 - ERROR - Traceback (most recent call last):
File "/contrail-test/tcutils/wrappers.py", line 80, in wrapper
result = function(self, *args, **kwargs)
File "/contrail-test/tcutils/util.py", line 1041, in wrapper
return f(self, *func_args, **func_kwargs)
File "/contrail-test/scripts/ecmp/test_ecmp.py", line 735, in test_svc_fate_sharing_basic
**self.common_args)
File "/contrail-test/common/servicechain/firewall/verify.py", line 160, in verify_multi_inline_svc_with_fate_share
ret_dict = self.config_multi_inline_svc(*args, si_list=si_list, **kwargs)
File "/contrail-test/common/servicechain/config.py", line 723, in config_multi_inline_svc
**kwargs)
File "/contrail-test/common/servicechain/config.py", line 533, in config_svc_chain
max_inst=max_inst)
File "/contrail-test/common/servicechain/config.py", line 335, in create_service_vms
zone=random.choice(valid_zones))
File "/contrail-test/common/servicechain/config.py", line 161, in config_vm_only
flavor=flavor, zone=zone, node_name=node_name,**kwargs)
File "/contrail-test/common/servicechain/config.py", line 203, in config_vm
flavor=flavor, vn_objs=vns, count=count, zone=zone,**kwargs))
File "/usr/local/lib/python3.6/site-packages/testtools/testcase.py", line 756, in useFixture
reraise(*exc_info)
File "/usr/local/lib/python3.6/site-packages/testtools/_compat3x.py", line 16, in reraise
raise exc_obj.with_traceback(exc_tb)
File "/usr/local/lib/python3.6/site-packages/testtools/testcase.py", line 731, in useFixture
fixture.setUp()
File "/contrail-test/fixtures/vm_test.py", line 188, in setUp
self.create()
File "/contrail-test/fixtures/vm_test.py", line 240, in create
fixed_ips=self.fixed_ips)
File "/contrail-test/fixtures/openstack.py", line 118, in create_vm
return self.nova_h.create_vm(vm_name=vm_name, image_name=image_name, vn_ids=vn_ids, **kwargs)
File "/contrail-test/fixtures/nova_test.py", line 580, in create_vm
image = self.get_image(image_name=image_name)
File "/contrail-test/fixtures/nova_test.py", line 199, in get_image
self._install_image(image_name=image_name)
File "/contrail-test/fixtures/nova_test.py", line 339, in _install_image
return self.copy_and_glance(build_path, image_name, params)
File "/contrail-test/fixtures/nova_test.py", line 410, in copy_and_glance
image_id = self.glance_h.create_image(generic_image_name, image_path_real, **params)
File "/contrail-test/fixtures/glance_test.py", line 50, in create_image
self.upload_image(obj['id'], filename)
File "/contrail-test/fixtures/glance_test.py", line 54, in upload_image
self.obj.images.upload(uuid, open(filename, 'rb'))
FileNotFoundError: [Errno 2] No such file or directory: '/contrail-test/images/tinycore-in-network.qcow2'
2025-07-23 15:17:32,662 - DEBUG - No XMPP flaps were noticed during the test
2025-07-23 15:17:32,662 - INFO -
2025-07-23 15:17:32,662 - INFO - END TEST : test_svc_fate_sharing_basic : FAILED[0:01:44]
2025-07-23 15:17:32,662 - INFO - --------------------------------------------------------------------------------
2025-07-23 15:17:33,994 - ERROR - TestFailed
Python 3.6.8: /usr/bin/python3
Wed Jul 23 15:17:32 2025
A problem occurred in a Python script. Here is the sequence of
function calls leading up to the error, in the order they occurred.
/contrail-test/tcutils/wrappers.py in wrapper(self=, *args=(), **kwargs={})
78 log.info('Initial checks done. Running the testcase now')
79 log.info('')
80 result = function(self, *args, **kwargs)
81 if self.inputs.upgrade:
82 pid = os.getpid()
result = None
function =
self =
args = ()
kwargs = {}
/contrail-test/scripts/ecmp/test_ecmp.py in test_svc_fate_sharing_basic(self=)
1198 @preposttest_wrapper
1199 def test_svc_fate_sharing_basic(self):
1200 super(TestMultiInlineSVCIPv6,self).test_svc_fate_sharing_basic()
1201
1202
builtinsuper =
global TestMultiInlineSVCIPv6 =
self =
).test_svc_fate_sharing_basic = >
/contrail-test/tcutils/wrappers.py in wrapper(self=, *args=(), **kwargs={})
174 if 'ci_image' in list(os.environ.keys()):
175 os.environ['stop_execution_flag'] = 'set'
176 raise TestFailed("\n ".join(errmsg))
177 elif testskip:
178 log.info('')
global TestFailed =
"\n ".join undefined
errmsg = ["Test failed: Traceback (most recent call la...il-test/images/tinycore-in-network.qcow2'\n
\n"]
TestFailed: Test failed: Traceback (most recent call last):
File "/contrail-test/tcutils/wrappers.py", line 80, in wrapper
result = function(self, *args, **kwargs)
File "/contrail-test/tcutils/util.py", line 1041, in wrapper
return f(self, *func_args, **func_kwargs)
File "/contrail-test/scripts/ecmp/test_ecmp.py", line 735, in test_svc_fate_sharing_basic
**self.common_args)
File "/contrail-test/common/servicechain/firewall/verify.py", line 160, in verify_multi_inline_svc_with_fate_share
ret_dict = self.config_multi_inline_svc(*args, si_list=si_list, **kwargs)
File "/contrail-test/common/servicechain/config.py", line 723, in config_multi_inline_svc
**kwargs)
File "/contrail-test/common/servicechain/config.py", line 533, in config_svc_chain
max_inst=max_inst)
File "/contrail-test/common/servicechain/config.py", line 335, in create_service_vms
zone=random.choice(valid_zones))
File "/contrail-test/common/servicechain/config.py", line 161, in config_vm_only
flavor=flavor, zone=zone, node_name=node_name,**kwargs)
File "/contrail-test/common/servicechain/config.py", line 203, in config_vm
flavor=flavor, vn_objs=vns, count=count, zone=zone,**kwargs))
File "/usr/local/lib/python3.6/site-packages/testtools/testcase.py", line 756, in useFixture
reraise(*exc_info)
File "/usr/local/lib/python3.6/site-packages/testtools/_compat3x.py", line 16, in reraise
raise exc_obj.with_traceback(exc_tb)
File "/usr/local/lib/python3.6/site-packages/testtools/testcase.py", line 731, in useFixture
fixture.setUp()
File "/contrail-test/fixtures/vm_test.py", line 188, in setUp
self.create()
File "/contrail-test/fixtures/vm_test.py", line 240, in create
fixed_ips=self.fixed_ips)
File "/contrail-test/fixtures/openstack.py", line 118, in create_vm
return self.nova_h.create_vm(vm_name=vm_name, image_name=image_name, vn_ids=vn_ids, **kwargs)
File "/contrail-test/fixtures/nova_test.py", line 580, in create_vm
image = self.get_image(image_name=image_name)
File "/contrail-test/fixtures/nova_test.py", line 199, in get_image
self._install_image(image_name=image_name)
File "/contrail-test/fixtures/nova_test.py", line 339, in _install_image
return self.copy_and_glance(build_path, image_name, params)
File "/contrail-test/fixtures/nova_test.py", line 410, in copy_and_glance
image_id = self.glance_h.create_image(generic_image_name, image_path_real, **params)
File "/contrail-test/fixtures/glance_test.py", line 50, in create_image
self.upload_image(obj['id'], filename)
File "/contrail-test/fixtures/glance_test.py", line 54, in upload_image
self.obj.images.upload(uuid, open(filename, 'rb'))
FileNotFoundError: [Errno 2] No such file or directory: '/contrail-test/images/tinycore-in-network.qcow2'
__cause__ = None
__class__ =
__context__ = None
__delattr__ =
__dict__ = {}
__dir__ =
__doc__ = None
__eq__ =
__format__ =
__ge__ =
__getattribute__ =
__gt__ =
__hash__ =
__init__ =
__init_subclass__ =
__le__ =
__lt__ =
__module__ = 'tcutils.cores'
__ne__ =
__new__ =
__reduce__ =
__reduce_ex__ =
__repr__ =
__setattr__ =
__setstate__ =
__sizeof__ =
__str__ =
__subclasshook__ =
__suppress_context__ = False
__traceback__ =
__weakref__ = None
args = ("Test failed: Traceback (most recent call la...il-test/images/tinycore-in-network.qcow2'\n
\n",)
with_traceback =
The above is a description of an error in a Python program. Here is
the original traceback:
Traceback (most recent call last):
File "/contrail-test/tcutils/wrappers.py", line 80, in wrapper
result = function(self, *args, **kwargs)
File "/contrail-test/scripts/ecmp/test_ecmp.py", line 1200, in test_svc_fate_sharing_basic
super(TestMultiInlineSVCIPv6,self).test_svc_fate_sharing_basic()
File "/contrail-test/tcutils/wrappers.py", line 176, in wrapper
raise TestFailed("\n ".join(errmsg))
tcutils.cores.TestFailed: Test failed: Traceback (most recent call last):
File "/contrail-test/tcutils/wrappers.py", line 80, in wrapper
result = function(self, *args, **kwargs)
File "/contrail-test/tcutils/util.py", line 1041, in wrapper
return f(self, *func_args, **func_kwargs)
File "/contrail-test/scripts/ecmp/test_ecmp.py", line 735, in test_svc_fate_sharing_basic
**self.common_args)
File "/contrail-test/common/servicechain/firewall/verify.py", line 160, in verify_multi_inline_svc_with_fate_share
ret_dict = self.config_multi_inline_svc(*args, si_list=si_list, **kwargs)
File "/contrail-test/common/servicechain/config.py", line 723, in config_multi_inline_svc
**kwargs)
File "/contrail-test/common/servicechain/config.py", line 533, in config_svc_chain
max_inst=max_inst)
File "/contrail-test/common/servicechain/config.py", line 335, in create_service_vms
zone=random.choice(valid_zones))
File "/contrail-test/common/servicechain/config.py", line 161, in config_vm_only
flavor=flavor, zone=zone, node_name=node_name,**kwargs)
File "/contrail-test/common/servicechain/config.py", line 203, in config_vm
flavor=flavor, vn_objs=vns, count=count, zone=zone,**kwargs))
File "/usr/local/lib/python3.6/site-packages/testtools/testcase.py", line 756, in useFixture
reraise(*exc_info)
File "/usr/local/lib/python3.6/site-packages/testtools/_compat3x.py", line 16, in reraise
raise exc_obj.with_traceback(exc_tb)
File "/usr/local/lib/python3.6/site-packages/testtools/testcase.py", line 731, in useFixture
fixture.setUp()
File "/contrail-test/fixtures/vm_test.py", line 188, in setUp
self.create()
File "/contrail-test/fixtures/vm_test.py", line 240, in create
fixed_ips=self.fixed_ips)
File "/contrail-test/fixtures/openstack.py", line 118, in create_vm
return self.nova_h.create_vm(vm_name=vm_name, image_name=image_name, vn_ids=vn_ids, **kwargs)
File "/contrail-test/fixtures/nova_test.py", line 580, in create_vm
image = self.get_image(image_name=image_name)
File "/contrail-test/fixtures/nova_test.py", line 199, in get_image
self._install_image(image_name=image_name)
File "/contrail-test/fixtures/nova_test.py", line 339, in _install_image
return self.copy_and_glance(build_path, image_name, params)
File "/contrail-test/fixtures/nova_test.py", line 410, in copy_and_glance
image_id = self.glance_h.create_image(generic_image_name, image_path_real, **params)
File "/contrail-test/fixtures/glance_test.py", line 50, in create_image
self.upload_image(obj['id'], filename)
File "/contrail-test/fixtures/glance_test.py", line 54, in upload_image
self.obj.images.upload(uuid, open(filename, 'rb'))
FileNotFoundError: [Errno 2] No such file or directory: '/contrail-test/images/tinycore-in-network.qcow2'
2025-07-23 15:17:34,027 - DEBUG - No XMPP flaps were noticed during the test
2025-07-23 15:17:34,027 - INFO -
2025-07-23 15:17:34,027 - INFO - END TEST : test_svc_fate_sharing_basic : FAILED[0:01:47]
2025-07-23 15:17:34,027 - INFO - --------------------------------------------------------------------------------
2025-07-23 15:17:34,030 - INFO - Skip interface_detach for VM ctest-right_vm_ctest-TestMultiInlineSVCIPv6-01153995-72096254 in state BUILD
2025-07-23 15:17:34,030 - INFO - Deleting VM ctest-right_vm_ctest-TestMultiInlineSVCIPv6-01153995-72096254
2025-07-23 15:17:34,061 - ERROR - VM ctest-right_vm_ctest-TestMultiInlineSVCIPv6-01153995-72096254 has failed to come up
2025-07-23 15:17:34,061 - ERROR - Nova failed to get host of the VM
2025-07-23 15:17:34,061 - ERROR - VM ctest-right_vm_ctest-TestMultiInlineSVCIPv6-01153995-72096254 has failed to come up
2025-07-23 15:17:34,061 - ERROR - Nova failed to get host of the VM
2025-07-23 15:17:34,129 - INFO - Skip interface_detach for VM ctest-left_vm_ctest-TestMultiInlineSVCIPv6-01153995-22458797 in state BUILD
2025-07-23 15:17:34,129 - INFO - Deleting VM ctest-left_vm_ctest-TestMultiInlineSVCIPv6-01153995-22458797
2025-07-23 15:17:34,129 - ERROR - VM ctest-left_vm_ctest-TestMultiInlineSVCIPv6-01153995-22458797 has failed to come up
2025-07-23 15:17:34,129 - ERROR - Nova failed to get host of the VM
2025-07-23 15:17:34,129 - ERROR - VM ctest-left_vm_ctest-TestMultiInlineSVCIPv6-01153995-22458797 has failed to come up
2025-07-23 15:17:34,130 - ERROR - Nova failed to get host of the VM
2025-07-23 15:17:34,191 - INFO - Deleting VN ctest-left_ctest-TestMultiInlineSVCIPv6-01153995-58755462
2025-07-23 15:17:34,246 - DEBUG - VN e3354eab-a43c-46b8-b28a-cdcdf0be97b9 still in use: Unable to complete operation on network e3354eab-a43c-46b8-b28a-cdcdf0be97b9. There are one or more ports still in use on the network.
Neutron server returns request_ids: ['req-1e878b18-d440-4518-973d-4d0973bf2328']
2025-07-23 15:17:34,246 - WARNING - Deleting VN ctest-left_ctest-TestMultiInlineSVCIPv6-01153995-58755462 failed..Will retry
2025-07-23 15:17:36,458 - DEBUG - Response for deleting network ()
2025-07-23 15:17:36,458 - INFO - Deleting VN ctest-right_ctest-TestMultiInlineSVCIPv6-01153995-16658227
2025-07-23 15:17:36,674 - DEBUG - Response for deleting network ()
2025-07-23 15:17:36,675 - INFO - Deleting VN ctest-mgmt_ctest-TestMultiInlineSVCIPv6-01153995-15875743
2025-07-23 15:17:36,890 - DEBUG - Response for deleting network ()
2025-07-23 15:17:37,707 - INFO - Deleted project: ctest-TestMultiInlineSVCIPv6-01153995, ID : 1b9b028e-e1b3-4aca-acc0-ca917cd5286d