2025-07-23 18:14:47,357 - INFO - Domain Default found not creating
2025-07-23 18:14:47,498 - INFO - Project ctest-FatFlowSerialIpv6-31491632 not found, creating it
2025-07-23 18:14:47,995 - INFO - Created Project:ctest-FatFlowSerialIpv6-31491632, ID : f6dd32c9-bf9c-4660-b7f6-0fc3933b4d03
2025-07-23 18:14:49,451 - INFO - ================================================================================
2025-07-23 18:14:49,451 - INFO - STARTING TEST : test_fat_flow_intra_vn_inter_node
2025-07-23 18:14:49,451 - INFO - TEST DESCRIPTION :
Description: Verify Fat flow for intra-VN inter-Node traffic
Steps:
1. create 1 VN and launch 3 VMs in it.client VMs on same node and server VM on different node.
2. on server VM, config Fat flow for udp port 53.
3. from both client VM, send UDP traffic to server on port 53 twice with diff. src ports
Pass criteria:
1. on client VMs compute, 4 set of flows and on server compute, 2 set of flows should be created
2. on server compute, flow's source port should be 0 for Fat flow
2025-07-23 18:14:50,773 - DEBUG - Skipping xmpp flap check
2025-07-23 18:14:50,773 - INFO - Initial checks done. Running the testcase now
2025-07-23 18:14:50,773 - INFO -
2025-07-23 18:14:51,472 - DEBUG - Response for create_network : {'network': {'id': '95c43151-b699-4ed8-9703-ff7b7f142752', 'name': 'ctest-vn-25477738', 'tenant_id': 'f6dd32c9bf9c4660b7f60fc3933b4d03', 'project_id': 'f6dd32c9bf9c4660b7f60fc3933b4d03', 'admin_state_up': True, 'shared': False, 'status': 'ACTIVE', 'router:external': False, 'mtu': 0, 'subnets': [], 'fq_name': ['default-domain', 'ctest-FatFlowSerialIpv6-31491632', 'ctest-vn-25477738'], 'port_security_enabled': True, 'description': ''}}
2025-07-23 18:14:51,741 - DEBUG - Response for create_subnet : {'subnet': {'id': '5467ce41-9389-4e58-b222-a49c0ec90976', 'name': '', 'tenant_id': 'f6dd32c9bf9c4660b7f60fc3933b4d03', 'network_id': '95c43151-b699-4ed8-9703-ff7b7f142752', 'ip_version': 4, 'cidr': '133.46.196.64/26', 'allocation_pools': [{'start': '133.46.196.66', 'end': '133.46.196.126'}], 'gateway_ip': '133.46.196.65', 'enable_dhcp': True, 'ipv6_ra_mode': None, 'ipv6_address_mode': None, 'dns_nameservers': [], 'host_routes': [], 'dns_server_address': '133.46.196.66', 'tags': [], 'project_id': 'f6dd32c9bf9c4660b7f60fc3933b4d03'}}
2025-07-23 18:14:51,961 - DEBUG - Response for create_subnet : {'subnet': {'id': '58dc40b9-3a1d-451a-a7fe-51e062d93c5e', 'name': '', 'tenant_id': 'f6dd32c9bf9c4660b7f60fc3933b4d03', 'network_id': '95c43151-b699-4ed8-9703-ff7b7f142752', 'ip_version': 6, 'cidr': '298d:cff:6d96:b8d2:fed4:6b94::/96', 'allocation_pools': [{'start': '298d:cff:6d96:b8d2:fed4:6b94:0:2', 'end': '298d:cff:6d96:b8d2:fed4:6b94:ffff:fffe'}], 'gateway_ip': '298d:cff:6d96:b8d2:fed4:6b94:0:1', 'enable_dhcp': True, 'ipv6_ra_mode': None, 'ipv6_address_mode': None, 'dns_nameservers': [], 'host_routes': [], 'dns_server_address': '298d:cff:6d96:b8d2:fed4:6b94:0:2', 'tags': [], 'project_id': 'f6dd32c9bf9c4660b7f60fc3933b4d03'}}
2025-07-23 18:14:51,989 - INFO - Created VN ctest-vn-25477738
2025-07-23 18:14:52,001 - DEBUG - VN ctest-vn-25477738 UUID is 95c43151-b699-4ed8-9703-ff7b7f142752
2025-07-23 18:14:52,101 - DEBUG - Requesting: http://10.0.0.23:8082/domains
2025-07-23 18:14:52,401 - DEBUG - Requesting: http://10.0.0.23:8082/domains
2025-07-23 18:14:52,428 - DEBUG - Requesting: http://10.0.0.23:8082/domain/85afe8be-97e7-4e27-9667-ee7272dde692
2025-07-23 18:14:52,444 - DEBUG - Requesting: http://10.0.0.23:8082/project/f6dd32c9-bf9c-4660-b7f6-0fc3933b4d03
2025-07-23 18:14:52,549 - DEBUG - Requesting: http://10.0.0.23:8082/virtual-network/95c43151-b699-4ed8-9703-ff7b7f142752
2025-07-23 18:14:52,563 - DEBUG - Requesting: http://10.0.0.23:8082/virtual-network/95c43151-b699-4ed8-9703-ff7b7f142752
2025-07-23 18:14:52,580 - DEBUG - Requesting: http://10.0.0.23:8082/routing-instance/71328e98-349c-445b-8863-2a11c939ca22
2025-07-23 18:14:52,592 - DEBUG - Requesting: http://10.0.0.23:8082/routing-instance/71328e98-349c-445b-8863-2a11c939ca22
2025-07-23 18:14:52,601 - DEBUG - Requesting: http://10.0.0.23:8082/route-target/35208473-a14a-4abe-abfa-a3d8f378182d
2025-07-23 18:14:52,609 - DEBUG - Route Targets: ['target:64512:8000005']
2025-07-23 18:14:52,609 - DEBUG - Requesting: http://10.0.0.23:8082/virtual-network/95c43151-b699-4ed8-9703-ff7b7f142752
2025-07-23 18:14:52,622 - DEBUG - Requesting: http://10.0.0.23:8082/routing-instance/71328e98-349c-445b-8863-2a11c939ca22
2025-07-23 18:14:52,639 - INFO - Verified VN network id 7 for VN 95c43151-b699-4ed8-9703-ff7b7f142752
2025-07-23 18:14:52,639 - INFO - Verifications in API Server for VN ctest-vn-25477738 passed
2025-07-23 18:14:52,639 - DEBUG - Requesting: http://10.0.0.23:8082/virtual-network/95c43151-b699-4ed8-9703-ff7b7f142752
2025-07-23 18:14:52,653 - DEBUG - Requesting: http://10.0.0.23:8082/routing-instance/71328e98-349c-445b-8863-2a11c939ca22
2025-07-23 18:14:52,662 - DEBUG - Requesting: http://10.0.0.23:8082/routing-instance/71328e98-349c-445b-8863-2a11c939ca22
2025-07-23 18:14:52,671 - DEBUG - Requesting: http://10.0.0.23:8082/route-target/35208473-a14a-4abe-abfa-a3d8f378182d
2025-07-23 18:14:52,691 - DEBUG - Control-node 10.0.0.23 : VN object is : {'node_name': 'virtual-network:default-domain:ctest-FatFlowSerialIpv6-31491632:ctest-vn-25477738', 'interests': '-', 'advertised': '-', 'dbentryflags': None, 'obj_info': {'sequence_number': '0', 'origin': 'Cassandra', 'data': {'id-perms': {'permissions': {'owner': 'ctest-FatFlowSerialIpv6-31491632', 'owner-access': '7', 'group': 'reader', 'group-access': '7', 'other-access': '7'}, 'uuid': {'uuid-mslong': '10791804834112163544', 'uuid-lslong': '10881822030535337810'}, 'enable': 'true', 'created': '2025-07-23T18:14:51', 'last-modified': '2025-07-23T18:14:51', 'description': None, 'user-visible': 'true', 'creator': None}}}, 'neighbors': None, 'last_modified': '00:00:00.772940'}
2025-07-23 18:14:52,702 - DEBUG - Route Targets: ['target:64512:8000005']
2025-07-23 18:14:52,709 - DEBUG - Control-node 10.0.0.249 : VN object is : {'node_name': 'virtual-network:default-domain:ctest-FatFlowSerialIpv6-31491632:ctest-vn-25477738', 'interests': '-', 'advertised': '-', 'dbentryflags': None, 'obj_info': {'sequence_number': '0', 'origin': 'Cassandra', 'data': {'id-perms': {'permissions': {'owner': 'ctest-FatFlowSerialIpv6-31491632', 'owner-access': '7', 'group': 'reader', 'group-access': '7', 'other-access': '7'}, 'uuid': {'uuid-mslong': '10791804834112163544', 'uuid-lslong': '10881822030535337810'}, 'enable': 'true', 'created': '2025-07-23T18:14:51', 'last-modified': '2025-07-23T18:14:51', 'description': None, 'user-visible': 'true', 'creator': None}}}, 'neighbors': None, 'last_modified': '00:00:00.788559'}
2025-07-23 18:14:52,717 - DEBUG - Route Targets: ['target:64512:8000005']
2025-07-23 18:14:52,723 - DEBUG - Control-node 10.0.0.73 : VN object is : {'node_name': 'virtual-network:default-domain:ctest-FatFlowSerialIpv6-31491632:ctest-vn-25477738', 'interests': '-', 'advertised': '-', 'dbentryflags': None, 'obj_info': {'sequence_number': '0', 'origin': 'Cassandra', 'data': {'id-perms': {'permissions': {'owner': 'ctest-FatFlowSerialIpv6-31491632', 'owner-access': '7', 'group': 'reader', 'group-access': '7', 'other-access': '7'}, 'uuid': {'uuid-mslong': '10791804834112163544', 'uuid-lslong': '10881822030535337810'}, 'enable': 'true', 'created': '2025-07-23T18:14:51', 'last-modified': '2025-07-23T18:14:51', 'description': None, 'user-visible': 'true', 'creator': None}}}, 'neighbors': None, 'last_modified': '00:00:00.805431'}
2025-07-23 18:14:52,730 - DEBUG - Route Targets: ['target:64512:8000005']
2025-07-23 18:14:52,730 - INFO - On all control nodes, Config, RI and RT verification for VN ctest-vn-25477738 passed
2025-07-23 18:14:52,730 - DEBUG - ====Verifying policy data for ctest-vn-25477738 in API_Server ======
2025-07-23 18:14:52,730 - DEBUG - Requesting: http://10.0.0.23:8082/domains
2025-07-23 18:14:52,738 - DEBUG - Requesting: http://10.0.0.23:8082/domain/85afe8be-97e7-4e27-9667-ee7272dde692
2025-07-23 18:14:52,752 - DEBUG - Requesting: http://10.0.0.23:8082/project/f6dd32c9-bf9c-4660-b7f6-0fc3933b4d03
2025-07-23 18:14:52,849 - DEBUG - Requesting: http://10.0.0.23:8082/virtual-network/95c43151-b699-4ed8-9703-ff7b7f142752
2025-07-23 18:14:52,861 - DEBUG - =>VN ctest-vn-25477738 has no policy to be verified
2025-07-23 18:14:52,861 - DEBUG - Verifying the vn in opserver
2025-07-23 18:14:52,861 - DEBUG - Verifying the default-domain:ctest-FatFlowSerialIpv6-31491632:ctest-vn-25477738 virtual network link through opserver 10.0.0.23
2025-07-23 18:14:52,861 - DEBUG - Requesting: http://10.0.0.23:8081/analytics/uves/virtual-networks
2025-07-23 18:14:53,152 - DEBUG - Requesting: http://10.0.0.23:8081/analytics/uves/virtual-networks
2025-07-23 18:14:53,210 - DEBUG - vn link and name as {'name': 'default-domain:ctest-FatFlowSerialIpv6-31491632:ctest-vn-25477738', 'href': 'http://10.0.0.23:8081/analytics/uves/virtual-network/default-domain:ctest-FatFlowSerialIpv6-31491632:ctest-vn-25477738?flat'}
2025-07-23 18:14:53,210 - DEBUG - Verifying the default-domain:ctest-FatFlowSerialIpv6-31491632:ctest-vn-25477738 virtual network link through opserver 10.0.0.249
2025-07-23 18:14:53,210 - DEBUG - Requesting: http://10.0.0.249:8081/analytics/uves/virtual-networks
2025-07-23 18:14:53,508 - DEBUG - Requesting: http://10.0.0.249:8081/analytics/uves/virtual-networks
2025-07-23 18:14:53,562 - DEBUG - vn link and name as {'name': 'default-domain:ctest-FatFlowSerialIpv6-31491632:ctest-vn-25477738', 'href': 'http://10.0.0.249:8081/analytics/uves/virtual-network/default-domain:ctest-FatFlowSerialIpv6-31491632:ctest-vn-25477738?flat'}
2025-07-23 18:14:53,562 - DEBUG - Verifying the default-domain:ctest-FatFlowSerialIpv6-31491632:ctest-vn-25477738 virtual network link through opserver 10.0.0.73
2025-07-23 18:14:53,562 - DEBUG - Requesting: http://10.0.0.73:8081/analytics/uves/virtual-networks
2025-07-23 18:14:53,859 - DEBUG - Requesting: http://10.0.0.73:8081/analytics/uves/virtual-networks
2025-07-23 18:14:53,912 - DEBUG - vn link and name as {'name': 'default-domain:ctest-FatFlowSerialIpv6-31491632:ctest-vn-25477738', 'href': 'http://10.0.0.73:8081/analytics/uves/virtual-network/default-domain:ctest-FatFlowSerialIpv6-31491632:ctest-vn-25477738?flat'}
2025-07-23 18:14:53,912 - INFO - Validated that VN default-domain:ctest-FatFlowSerialIpv6-31491632:ctest-vn-25477738 is found in opserver
2025-07-23 18:14:53,949 - DEBUG - Do not have enough data to verify VN in agent
2025-07-23 18:14:53,967 - DEBUG - VRF ids for VN ctest-vn-25477738: {}
2025-07-23 18:14:53,988 - DEBUG - Services list from nova: [Traceback (most recent call last):
File "/contrail-test/tcutils/wrappers.py", line 80, in wrapper
result = function(self, *args, **kwargs)
File "/contrail-test/serial_scripts/vrouter/test_fat_flow_serial.py", line 41, in test_fat_flow_intra_vn_inter_node
node_name=compute_hosts[0])
File "/contrail-test/common/vrouter/base.py", line 179, in create_vms
*args, **kwargs
File "/contrail-test/common/base.py", line 281, in create_vm
**kwargs)
File "/contrail-test/common/base.py", line 254, in create_only_vm
vm_obj.setUp()
File "/contrail-test/fixtures/vm_test.py", line 188, in setUp
self.create()
File "/contrail-test/fixtures/vm_test.py", line 240, in create
fixed_ips=self.fixed_ips)
File "/contrail-test/fixtures/openstack.py", line 118, in create_vm
return self.nova_h.create_vm(vm_name=vm_name, image_name=image_name, vn_ids=vn_ids, **kwargs)
File "/contrail-test/fixtures/nova_test.py", line 580, in create_vm
image = self.get_image(image_name=image_name)
File "/contrail-test/fixtures/nova_test.py", line 199, in get_image
self._install_image(image_name=image_name)
File "/contrail-test/fixtures/nova_test.py", line 339, in _install_image
return self.copy_and_glance(build_path, image_name, params)
File "/contrail-test/fixtures/nova_test.py", line 410, in copy_and_glance
image_id = self.glance_h.create_image(generic_image_name, image_path_real, **params)
File "/contrail-test/fixtures/glance_test.py", line 50, in create_image
self.upload_image(obj['id'], filename)
File "/contrail-test/fixtures/glance_test.py", line 54, in upload_image
self.obj.images.upload(uuid, open(filename, 'rb'))
FileNotFoundError: [Errno 2] No such file or directory: '/contrail-test/images/ubuntu.vmdk'
2025-07-23 18:16:36,201 - DEBUG - Skipping xmpp flap check
2025-07-23 18:16:36,201 - INFO -