]) created on node: (an-jenkins-deploy-platform-ansible-os-3693-1), Zone: (nova:an-jenkins-deploy-platform-ansible-os-3693-1)
2025-07-22 15:25:12,852 - INFO - ================================================================================
2025-07-22 15:25:12,852 - INFO - STARTING TEST : test_svc_fate_sharing_basic
2025-07-22 15:25:14,199 - DEBUG - Nothing to compare xmpp stats {'10.0.0.19': {'10.20.0.14': '0', '10.20.0.17': '0'}, '10.0.0.27': {'10.20.0.14': '0', '10.20.0.254': '0'}} with
2025-07-22 15:25:14,199 - INFO - Initial checks done. Running the testcase now
2025-07-22 15:25:14,199 - INFO -
2025-07-22 15:25:14,199 - INFO - ================================================================================
2025-07-22 15:25:14,199 - INFO - STARTING TEST : test_svc_fate_sharing_basic
2025-07-22 15:25:14,199 - INFO - TEST DESCRIPTION :
Description: Basic test to validate fate sharing in a multi inline service chain with 2 SIs, 1 SVM each.
Test steps:
1.Creating vm's - vm1 and vm2 in networks vn1 and vn2.
2.Creating 2 inline service instances SI1 and SI2.
3.Creating a service chain by applying the 2 service instances in a policy between t
he VNs.
4.Associate an HC instance with right intf of one of the SVMs
5.Bring down the intf in the step 4
6.Verify that all the re-origined routes(ServiceChain) are deleted from all the RIs of all the SIs
7.Verify the step 6 in both control node and agent
8.Bring up the intf in the step 5
8.Verify that the routes should be re-originated again, and the traffic also starts flowing
Pass criteria: Ping/Route deletion/Route addition should be successful.
Maintainer : ankitja@juniper.net
2025-07-22 15:25:15,510 - DEBUG - Nothing to compare xmpp stats {'10.0.0.19': {'10.20.0.14': '0', '10.20.0.17': '0'}, '10.0.0.27': {'10.20.0.14': '0', '10.20.0.254': '0'}} with
2025-07-22 15:25:15,510 - INFO - Initial checks done. Running the testcase now
2025-07-22 15:25:15,511 - INFO -
2025-07-22 15:25:15,511 - DEBUG - Hosts: {'nova': ['an-jenkins-deploy-platform-ansible-os-3693-2', 'an-jenkins-deploy-platform-ansible-os-3693-1']}
2025-07-22 15:25:15,511 - DEBUG - Creating service template: ['default-domain', 'ctest-service_template_1-47585668']
2025-07-22 15:25:15,830 - DEBUG - Service template: ['default-domain', 'ctest-service_template_1-47585668'] created succesfully
2025-07-22 15:26:56,428 - DEBUG - Installing image tiny_in_net
2025-07-22 15:26:56,428 - DEBUG - Image info {'name': 'tinycore-in-network.qcow2.gz', 'webserver': '', 'location': '/images/tinycore/', 'username': 'tc', 'password': 'secret', 'type': 'os', 'flavor': 'contrail_flavor_tiny', 'params': '--container-format bare --disk-format vmdk --property vmware_disktype="sparse" --property vmware_adaptertype="ide" --property hypervisor_type=qemu', 'name_docker': 'phusion-baseimage-enablesshd', 'vmdk': 'service_chain_images/nat_service/tiny_in_net/tinycore-in-network-flat.vmdk', 'shrinked_vmdk': "'no'"}
2025-07-22 15:26:56,428 - DEBUG - Image build path file:///contrail-test/images/tinycore-in-network.qcow2.gz
2025-07-22 15:26:56,428 - DEBUG - Download image from local file file:///contrail-test/images/tinycore-in-network.qcow2.gz
2025-07-22 15:26:56,428 - DEBUG - Unzip image
2025-07-22 15:26:56,428 - DEBUG - Image download (local=True) (proxy=None): Execute cmd: gunzip -f /contrail-test/images/tinycore-in-network.qcow2.gz
2025-07-22 15:26:56,442 - DEBUG - Try to create image in user context
2025-07-22 15:26:56,506 - DEBUG - Deleting service template: ['default-domain', 'ctest-service_template_1-47585668']
2025-07-22 15:26:56,600 - INFO - Service template: ['default-domain', 'ctest-service_template_1-47585668'] deleted successfully.
2025-07-22 15:26:57,884 - ERROR - Traceback (most recent call last):
File "/contrail-test/tcutils/wrappers.py", line 80, in wrapper
result = function(self, *args, **kwargs)
File "/contrail-test/tcutils/util.py", line 1041, in wrapper
return f(self, *func_args, **func_kwargs)
File "/contrail-test/scripts/ecmp/test_ecmp.py", line 735, in test_svc_fate_sharing_basic
**self.common_args)
File "/contrail-test/common/servicechain/firewall/verify.py", line 160, in verify_multi_inline_svc_with_fate_share
ret_dict = self.config_multi_inline_svc(*args, si_list=si_list, **kwargs)
File "/contrail-test/common/servicechain/config.py", line 723, in config_multi_inline_svc
**kwargs)
File "/contrail-test/common/servicechain/config.py", line 533, in config_svc_chain
max_inst=max_inst)
File "/contrail-test/common/servicechain/config.py", line 335, in create_service_vms
zone=random.choice(valid_zones))
File "/contrail-test/common/servicechain/config.py", line 161, in config_vm_only
flavor=flavor, zone=zone, node_name=node_name,**kwargs)
File "/contrail-test/common/servicechain/config.py", line 203, in config_vm
flavor=flavor, vn_objs=vns, count=count, zone=zone,**kwargs))
File "/usr/local/lib/python3.6/site-packages/testtools/testcase.py", line 756, in useFixture
reraise(*exc_info)
File "/usr/local/lib/python3.6/site-packages/testtools/_compat3x.py", line 16, in reraise
raise exc_obj.with_traceback(exc_tb)
File "/usr/local/lib/python3.6/site-packages/testtools/testcase.py", line 731, in useFixture
fixture.setUp()
File "/contrail-test/fixtures/vm_test.py", line 188, in setUp
self.create()
File "/contrail-test/fixtures/vm_test.py", line 240, in create
fixed_ips=self.fixed_ips)
File "/contrail-test/fixtures/openstack.py", line 118, in create_vm
return self.nova_h.create_vm(vm_name=vm_name, image_name=image_name, vn_ids=vn_ids, **kwargs)
File "/contrail-test/fixtures/nova_test.py", line 577, in create_vm
image = self.get_image(image_name=image_name)
File "/contrail-test/fixtures/nova_test.py", line 198, in get_image
self._install_image(image_name=image_name)
File "/contrail-test/fixtures/nova_test.py", line 338, in _install_image
return self.copy_and_glance(build_path, image_name, params)
File "/contrail-test/fixtures/nova_test.py", line 407, in copy_and_glance
image_id = self.glance_h.create_image(generic_image_name, image_path_real, **params)
File "/contrail-test/fixtures/glance_test.py", line 50, in create_image
self.upload_image(obj['id'], filename)
File "/contrail-test/fixtures/glance_test.py", line 54, in upload_image
self.obj.images.upload(uuid, open(filename, 'rb'))
FileNotFoundError: [Errno 2] No such file or directory: '/contrail-test/images/tinycore-in-network.qcow2'
2025-07-22 15:26:57,900 - DEBUG - No XMPP flaps were noticed during the test
2025-07-22 15:26:57,901 - INFO -
2025-07-22 15:26:57,901 - INFO - END TEST : test_svc_fate_sharing_basic : FAILED[0:01:43]
2025-07-22 15:26:57,901 - INFO - --------------------------------------------------------------------------------
2025-07-22 15:26:59,173 - ERROR - TestFailed
Python 3.6.8: /usr/bin/python3
Tue Jul 22 15:26:57 2025
A problem occurred in a Python script. Here is the sequence of
function calls leading up to the error, in the order they occurred.
/contrail-test/tcutils/wrappers.py in wrapper(self=, *args=(), **kwargs={})
78 log.info('Initial checks done. Running the testcase now')
79 log.info('')
80 result = function(self, *args, **kwargs)
81 if self.inputs.upgrade:
82 pid = os.getpid()
result = None
function =
self =
args = ()
kwargs = {}
/contrail-test/scripts/ecmp/test_ecmp.py in test_svc_fate_sharing_basic(self=)
1198 @preposttest_wrapper
1199 def test_svc_fate_sharing_basic(self):
1200 super(TestMultiInlineSVCIPv6,self).test_svc_fate_sharing_basic()
1201
1202
builtinsuper =
global TestMultiInlineSVCIPv6 =
self =
).test_svc_fate_sharing_basic = >
/contrail-test/tcutils/wrappers.py in wrapper(self=, *args=(), **kwargs={})
174 if 'ci_image' in list(os.environ.keys()):
175 os.environ['stop_execution_flag'] = 'set'
176 raise TestFailed("\n ".join(errmsg))
177 elif testskip:
178 log.info('')
global TestFailed =
"\n ".join undefined
errmsg = ["Test failed: Traceback (most recent call la...il-test/images/tinycore-in-network.qcow2'\n
\n"]
TestFailed: Test failed: Traceback (most recent call last):
File "/contrail-test/tcutils/wrappers.py", line 80, in wrapper
result = function(self, *args, **kwargs)
File "/contrail-test/tcutils/util.py", line 1041, in wrapper
return f(self, *func_args, **func_kwargs)
File "/contrail-test/scripts/ecmp/test_ecmp.py", line 735, in test_svc_fate_sharing_basic
**self.common_args)
File "/contrail-test/common/servicechain/firewall/verify.py", line 160, in verify_multi_inline_svc_with_fate_share
ret_dict = self.config_multi_inline_svc(*args, si_list=si_list, **kwargs)
File "/contrail-test/common/servicechain/config.py", line 723, in config_multi_inline_svc
**kwargs)
File "/contrail-test/common/servicechain/config.py", line 533, in config_svc_chain
max_inst=max_inst)
File "/contrail-test/common/servicechain/config.py", line 335, in create_service_vms
zone=random.choice(valid_zones))
File "/contrail-test/common/servicechain/config.py", line 161, in config_vm_only
flavor=flavor, zone=zone, node_name=node_name,**kwargs)
File "/contrail-test/common/servicechain/config.py", line 203, in config_vm
flavor=flavor, vn_objs=vns, count=count, zone=zone,**kwargs))
File "/usr/local/lib/python3.6/site-packages/testtools/testcase.py", line 756, in useFixture
reraise(*exc_info)
File "/usr/local/lib/python3.6/site-packages/testtools/_compat3x.py", line 16, in reraise
raise exc_obj.with_traceback(exc_tb)
File "/usr/local/lib/python3.6/site-packages/testtools/testcase.py", line 731, in useFixture
fixture.setUp()
File "/contrail-test/fixtures/vm_test.py", line 188, in setUp
self.create()
File "/contrail-test/fixtures/vm_test.py", line 240, in create
fixed_ips=self.fixed_ips)
File "/contrail-test/fixtures/openstack.py", line 118, in create_vm
return self.nova_h.create_vm(vm_name=vm_name, image_name=image_name, vn_ids=vn_ids, **kwargs)
File "/contrail-test/fixtures/nova_test.py", line 577, in create_vm
image = self.get_image(image_name=image_name)
File "/contrail-test/fixtures/nova_test.py", line 198, in get_image
self._install_image(image_name=image_name)
File "/contrail-test/fixtures/nova_test.py", line 338, in _install_image
return self.copy_and_glance(build_path, image_name, params)
File "/contrail-test/fixtures/nova_test.py", line 407, in copy_and_glance
image_id = self.glance_h.create_image(generic_image_name, image_path_real, **params)
File "/contrail-test/fixtures/glance_test.py", line 50, in create_image
self.upload_image(obj['id'], filename)
File "/contrail-test/fixtures/glance_test.py", line 54, in upload_image
self.obj.images.upload(uuid, open(filename, 'rb'))
FileNotFoundError: [Errno 2] No such file or directory: '/contrail-test/images/tinycore-in-network.qcow2'
__cause__ = None
__class__ =
__context__ = None
__delattr__ =
__dict__ = {}
__dir__ =
__doc__ = None
__eq__ =
__format__ =
__ge__ =
__getattribute__ =
__gt__ =
__hash__ =
__init__ =
__init_subclass__ =
__le__ =
__lt__ =
__module__ = 'tcutils.cores'
__ne__ =
__new__ =
__reduce__ =
__reduce_ex__ =
__repr__ =
__setattr__ =
__setstate__ =
__sizeof__ =
__str__ =
__subclasshook__ =
__suppress_context__ = False
__traceback__ =
__weakref__ = None
args = ("Test failed: Traceback (most recent call la...il-test/images/tinycore-in-network.qcow2'\n
\n",)
with_traceback =
The above is a description of an error in a Python program. Here is
the original traceback:
Traceback (most recent call last):
File "/contrail-test/tcutils/wrappers.py", line 80, in wrapper
result = function(self, *args, **kwargs)
File "/contrail-test/scripts/ecmp/test_ecmp.py", line 1200, in test_svc_fate_sharing_basic
super(TestMultiInlineSVCIPv6,self).test_svc_fate_sharing_basic()
File "/contrail-test/tcutils/wrappers.py", line 176, in wrapper
raise TestFailed("\n ".join(errmsg))
tcutils.cores.TestFailed: Test failed: Traceback (most recent call last):
File "/contrail-test/tcutils/wrappers.py", line 80, in wrapper
result = function(self, *args, **kwargs)
File "/contrail-test/tcutils/util.py", line 1041, in wrapper
return f(self, *func_args, **func_kwargs)
File "/contrail-test/scripts/ecmp/test_ecmp.py", line 735, in test_svc_fate_sharing_basic
**self.common_args)
File "/contrail-test/common/servicechain/firewall/verify.py", line 160, in verify_multi_inline_svc_with_fate_share
ret_dict = self.config_multi_inline_svc(*args, si_list=si_list, **kwargs)
File "/contrail-test/common/servicechain/config.py", line 723, in config_multi_inline_svc
**kwargs)
File "/contrail-test/common/servicechain/config.py", line 533, in config_svc_chain
max_inst=max_inst)
File "/contrail-test/common/servicechain/config.py", line 335, in create_service_vms
zone=random.choice(valid_zones))
File "/contrail-test/common/servicechain/config.py", line 161, in config_vm_only
flavor=flavor, zone=zone, node_name=node_name,**kwargs)
File "/contrail-test/common/servicechain/config.py", line 203, in config_vm
flavor=flavor, vn_objs=vns, count=count, zone=zone,**kwargs))
File "/usr/local/lib/python3.6/site-packages/testtools/testcase.py", line 756, in useFixture
reraise(*exc_info)
File "/usr/local/lib/python3.6/site-packages/testtools/_compat3x.py", line 16, in reraise
raise exc_obj.with_traceback(exc_tb)
File "/usr/local/lib/python3.6/site-packages/testtools/testcase.py", line 731, in useFixture
fixture.setUp()
File "/contrail-test/fixtures/vm_test.py", line 188, in setUp
self.create()
File "/contrail-test/fixtures/vm_test.py", line 240, in create
fixed_ips=self.fixed_ips)
File "/contrail-test/fixtures/openstack.py", line 118, in create_vm
return self.nova_h.create_vm(vm_name=vm_name, image_name=image_name, vn_ids=vn_ids, **kwargs)
File "/contrail-test/fixtures/nova_test.py", line 577, in create_vm
image = self.get_image(image_name=image_name)
File "/contrail-test/fixtures/nova_test.py", line 198, in get_image
self._install_image(image_name=image_name)
File "/contrail-test/fixtures/nova_test.py", line 338, in _install_image
return self.copy_and_glance(build_path, image_name, params)
File "/contrail-test/fixtures/nova_test.py", line 407, in copy_and_glance
image_id = self.glance_h.create_image(generic_image_name, image_path_real, **params)
File "/contrail-test/fixtures/glance_test.py", line 50, in create_image
self.upload_image(obj['id'], filename)
File "/contrail-test/fixtures/glance_test.py", line 54, in upload_image
self.obj.images.upload(uuid, open(filename, 'rb'))
FileNotFoundError: [Errno 2] No such file or directory: '/contrail-test/images/tinycore-in-network.qcow2'
2025-07-22 15:26:59,191 - DEBUG - No XMPP flaps were noticed during the test
2025-07-22 15:26:59,191 - INFO -
2025-07-22 15:26:59,191 - INFO - END TEST : test_svc_fate_sharing_basic : FAILED[0:01:47]
2025-07-22 15:26:59,191 - INFO - --------------------------------------------------------------------------------
2025-07-22 15:26:59,193 - INFO - Skip interface_detach for VM ctest-right_vm_ctest-TestMultiInlineSVCIPv6-01336199-90647831 in state BUILD
2025-07-22 15:26:59,194 - INFO - Deleting VM ctest-right_vm_ctest-TestMultiInlineSVCIPv6-01336199-90647831
2025-07-22 15:26:59,221 - ERROR - VM ctest-right_vm_ctest-TestMultiInlineSVCIPv6-01336199-90647831 has failed to come up
2025-07-22 15:26:59,221 - ERROR - Nova failed to get host of the VM
2025-07-22 15:26:59,221 - ERROR - VM ctest-right_vm_ctest-TestMultiInlineSVCIPv6-01336199-90647831 has failed to come up
2025-07-22 15:26:59,221 - ERROR - Nova failed to get host of the VM
2025-07-22 15:26:59,294 - INFO - Skip interface_detach for VM ctest-left_vm_ctest-TestMultiInlineSVCIPv6-01336199-75499456 in state BUILD
2025-07-22 15:26:59,294 - INFO - Deleting VM ctest-left_vm_ctest-TestMultiInlineSVCIPv6-01336199-75499456
2025-07-22 15:26:59,294 - ERROR - VM ctest-left_vm_ctest-TestMultiInlineSVCIPv6-01336199-75499456 has failed to come up
2025-07-22 15:26:59,294 - ERROR - Nova failed to get host of the VM
2025-07-22 15:26:59,294 - ERROR - VM ctest-left_vm_ctest-TestMultiInlineSVCIPv6-01336199-75499456 has failed to come up
2025-07-22 15:26:59,294 - ERROR - Nova failed to get host of the VM
2025-07-22 15:26:59,396 - INFO - Deleting VN ctest-left_ctest-TestMultiInlineSVCIPv6-01336199-65812727
2025-07-22 15:26:59,445 - DEBUG - VN 5f9658b9-582f-4158-8c97-2da29c15cd33 still in use: Unable to complete operation on network 5f9658b9-582f-4158-8c97-2da29c15cd33. There are one or more ports still in use on the network.
Neutron server returns request_ids: ['req-f8412e69-48ad-422b-bf4c-cdb8b003c46c']
2025-07-22 15:26:59,445 - WARNING - Deleting VN ctest-left_ctest-TestMultiInlineSVCIPv6-01336199-65812727 failed..Will retry
2025-07-22 15:27:01,698 - DEBUG - Response for deleting network ()
2025-07-22 15:27:01,698 - INFO - Deleting VN ctest-right_ctest-TestMultiInlineSVCIPv6-01336199-49147670
2025-07-22 15:27:01,919 - DEBUG - Response for deleting network ()
2025-07-22 15:27:01,919 - INFO - Deleting VN ctest-mgmt_ctest-TestMultiInlineSVCIPv6-01336199-63686538
2025-07-22 15:27:02,124 - DEBUG - Response for deleting network ()
2025-07-22 15:27:02,959 - INFO - Deleted project: ctest-TestMultiInlineSVCIPv6-01336199, ID : 2f07d748-f4d2-4575-8bc9-8db06a08d0f5