2025-07-23 18:14:47,357 - INFO - Domain Default found not creating 2025-07-23 18:14:47,498 - INFO - Project ctest-FatFlowSerialIpv6-31491632 not found, creating it 2025-07-23 18:14:47,995 - INFO - Created Project:ctest-FatFlowSerialIpv6-31491632, ID : f6dd32c9-bf9c-4660-b7f6-0fc3933b4d03 2025-07-23 18:14:49,451 - INFO - ================================================================================ 2025-07-23 18:14:49,451 - INFO - STARTING TEST : test_fat_flow_intra_vn_inter_node 2025-07-23 18:14:49,451 - INFO - TEST DESCRIPTION : Description: Verify Fat flow for intra-VN inter-Node traffic Steps: 1. create 1 VN and launch 3 VMs in it.client VMs on same node and server VM on different node. 2. on server VM, config Fat flow for udp port 53. 3. from both client VM, send UDP traffic to server on port 53 twice with diff. src ports Pass criteria: 1. on client VMs compute, 4 set of flows and on server compute, 2 set of flows should be created 2. on server compute, flow's source port should be 0 for Fat flow 2025-07-23 18:14:50,773 - DEBUG - Skipping xmpp flap check 2025-07-23 18:14:50,773 - INFO - Initial checks done. Running the testcase now 2025-07-23 18:14:50,773 - INFO - 2025-07-23 18:14:51,472 - DEBUG - Response for create_network : {'network': {'id': '95c43151-b699-4ed8-9703-ff7b7f142752', 'name': 'ctest-vn-25477738', 'tenant_id': 'f6dd32c9bf9c4660b7f60fc3933b4d03', 'project_id': 'f6dd32c9bf9c4660b7f60fc3933b4d03', 'admin_state_up': True, 'shared': False, 'status': 'ACTIVE', 'router:external': False, 'mtu': 0, 'subnets': [], 'fq_name': ['default-domain', 'ctest-FatFlowSerialIpv6-31491632', 'ctest-vn-25477738'], 'port_security_enabled': True, 'description': ''}} 2025-07-23 18:14:51,741 - DEBUG - Response for create_subnet : {'subnet': {'id': '5467ce41-9389-4e58-b222-a49c0ec90976', 'name': '', 'tenant_id': 'f6dd32c9bf9c4660b7f60fc3933b4d03', 'network_id': '95c43151-b699-4ed8-9703-ff7b7f142752', 'ip_version': 4, 'cidr': '133.46.196.64/26', 'allocation_pools': [{'start': '133.46.196.66', 'end': '133.46.196.126'}], 'gateway_ip': '133.46.196.65', 'enable_dhcp': True, 'ipv6_ra_mode': None, 'ipv6_address_mode': None, 'dns_nameservers': [], 'host_routes': [], 'dns_server_address': '133.46.196.66', 'tags': [], 'project_id': 'f6dd32c9bf9c4660b7f60fc3933b4d03'}} 2025-07-23 18:14:51,961 - DEBUG - Response for create_subnet : {'subnet': {'id': '58dc40b9-3a1d-451a-a7fe-51e062d93c5e', 'name': '', 'tenant_id': 'f6dd32c9bf9c4660b7f60fc3933b4d03', 'network_id': '95c43151-b699-4ed8-9703-ff7b7f142752', 'ip_version': 6, 'cidr': '298d:cff:6d96:b8d2:fed4:6b94::/96', 'allocation_pools': [{'start': '298d:cff:6d96:b8d2:fed4:6b94:0:2', 'end': '298d:cff:6d96:b8d2:fed4:6b94:ffff:fffe'}], 'gateway_ip': '298d:cff:6d96:b8d2:fed4:6b94:0:1', 'enable_dhcp': True, 'ipv6_ra_mode': None, 'ipv6_address_mode': None, 'dns_nameservers': [], 'host_routes': [], 'dns_server_address': '298d:cff:6d96:b8d2:fed4:6b94:0:2', 'tags': [], 'project_id': 'f6dd32c9bf9c4660b7f60fc3933b4d03'}} 2025-07-23 18:14:51,989 - INFO - Created VN ctest-vn-25477738 2025-07-23 18:14:52,001 - DEBUG - VN ctest-vn-25477738 UUID is 95c43151-b699-4ed8-9703-ff7b7f142752 2025-07-23 18:14:52,101 - DEBUG - Requesting: http://10.0.0.23:8082/domains 2025-07-23 18:14:52,401 - DEBUG - Requesting: http://10.0.0.23:8082/domains 2025-07-23 18:14:52,428 - DEBUG - Requesting: http://10.0.0.23:8082/domain/85afe8be-97e7-4e27-9667-ee7272dde692 2025-07-23 18:14:52,444 - DEBUG - Requesting: http://10.0.0.23:8082/project/f6dd32c9-bf9c-4660-b7f6-0fc3933b4d03 2025-07-23 18:14:52,549 - DEBUG - Requesting: http://10.0.0.23:8082/virtual-network/95c43151-b699-4ed8-9703-ff7b7f142752 2025-07-23 18:14:52,563 - DEBUG - Requesting: http://10.0.0.23:8082/virtual-network/95c43151-b699-4ed8-9703-ff7b7f142752 2025-07-23 18:14:52,580 - DEBUG - Requesting: http://10.0.0.23:8082/routing-instance/71328e98-349c-445b-8863-2a11c939ca22 2025-07-23 18:14:52,592 - DEBUG - Requesting: http://10.0.0.23:8082/routing-instance/71328e98-349c-445b-8863-2a11c939ca22 2025-07-23 18:14:52,601 - DEBUG - Requesting: http://10.0.0.23:8082/route-target/35208473-a14a-4abe-abfa-a3d8f378182d 2025-07-23 18:14:52,609 - DEBUG - Route Targets: ['target:64512:8000005'] 2025-07-23 18:14:52,609 - DEBUG - Requesting: http://10.0.0.23:8082/virtual-network/95c43151-b699-4ed8-9703-ff7b7f142752 2025-07-23 18:14:52,622 - DEBUG - Requesting: http://10.0.0.23:8082/routing-instance/71328e98-349c-445b-8863-2a11c939ca22 2025-07-23 18:14:52,639 - INFO - Verified VN network id 7 for VN 95c43151-b699-4ed8-9703-ff7b7f142752 2025-07-23 18:14:52,639 - INFO - Verifications in API Server for VN ctest-vn-25477738 passed 2025-07-23 18:14:52,639 - DEBUG - Requesting: http://10.0.0.23:8082/virtual-network/95c43151-b699-4ed8-9703-ff7b7f142752 2025-07-23 18:14:52,653 - DEBUG - Requesting: http://10.0.0.23:8082/routing-instance/71328e98-349c-445b-8863-2a11c939ca22 2025-07-23 18:14:52,662 - DEBUG - Requesting: http://10.0.0.23:8082/routing-instance/71328e98-349c-445b-8863-2a11c939ca22 2025-07-23 18:14:52,671 - DEBUG - Requesting: http://10.0.0.23:8082/route-target/35208473-a14a-4abe-abfa-a3d8f378182d 2025-07-23 18:14:52,691 - DEBUG - Control-node 10.0.0.23 : VN object is : {'node_name': 'virtual-network:default-domain:ctest-FatFlowSerialIpv6-31491632:ctest-vn-25477738', 'interests': '-', 'advertised': '-', 'dbentryflags': None, 'obj_info': {'sequence_number': '0', 'origin': 'Cassandra', 'data': {'id-perms': {'permissions': {'owner': 'ctest-FatFlowSerialIpv6-31491632', 'owner-access': '7', 'group': 'reader', 'group-access': '7', 'other-access': '7'}, 'uuid': {'uuid-mslong': '10791804834112163544', 'uuid-lslong': '10881822030535337810'}, 'enable': 'true', 'created': '2025-07-23T18:14:51', 'last-modified': '2025-07-23T18:14:51', 'description': None, 'user-visible': 'true', 'creator': None}}}, 'neighbors': None, 'last_modified': '00:00:00.772940'} 2025-07-23 18:14:52,702 - DEBUG - Route Targets: ['target:64512:8000005'] 2025-07-23 18:14:52,709 - DEBUG - Control-node 10.0.0.249 : VN object is : {'node_name': 'virtual-network:default-domain:ctest-FatFlowSerialIpv6-31491632:ctest-vn-25477738', 'interests': '-', 'advertised': '-', 'dbentryflags': None, 'obj_info': {'sequence_number': '0', 'origin': 'Cassandra', 'data': {'id-perms': {'permissions': {'owner': 'ctest-FatFlowSerialIpv6-31491632', 'owner-access': '7', 'group': 'reader', 'group-access': '7', 'other-access': '7'}, 'uuid': {'uuid-mslong': '10791804834112163544', 'uuid-lslong': '10881822030535337810'}, 'enable': 'true', 'created': '2025-07-23T18:14:51', 'last-modified': '2025-07-23T18:14:51', 'description': None, 'user-visible': 'true', 'creator': None}}}, 'neighbors': None, 'last_modified': '00:00:00.788559'} 2025-07-23 18:14:52,717 - DEBUG - Route Targets: ['target:64512:8000005'] 2025-07-23 18:14:52,723 - DEBUG - Control-node 10.0.0.73 : VN object is : {'node_name': 'virtual-network:default-domain:ctest-FatFlowSerialIpv6-31491632:ctest-vn-25477738', 'interests': '-', 'advertised': '-', 'dbentryflags': None, 'obj_info': {'sequence_number': '0', 'origin': 'Cassandra', 'data': {'id-perms': {'permissions': {'owner': 'ctest-FatFlowSerialIpv6-31491632', 'owner-access': '7', 'group': 'reader', 'group-access': '7', 'other-access': '7'}, 'uuid': {'uuid-mslong': '10791804834112163544', 'uuid-lslong': '10881822030535337810'}, 'enable': 'true', 'created': '2025-07-23T18:14:51', 'last-modified': '2025-07-23T18:14:51', 'description': None, 'user-visible': 'true', 'creator': None}}}, 'neighbors': None, 'last_modified': '00:00:00.805431'} 2025-07-23 18:14:52,730 - DEBUG - Route Targets: ['target:64512:8000005'] 2025-07-23 18:14:52,730 - INFO - On all control nodes, Config, RI and RT verification for VN ctest-vn-25477738 passed 2025-07-23 18:14:52,730 - DEBUG - ====Verifying policy data for ctest-vn-25477738 in API_Server ====== 2025-07-23 18:14:52,730 - DEBUG - Requesting: http://10.0.0.23:8082/domains 2025-07-23 18:14:52,738 - DEBUG - Requesting: http://10.0.0.23:8082/domain/85afe8be-97e7-4e27-9667-ee7272dde692 2025-07-23 18:14:52,752 - DEBUG - Requesting: http://10.0.0.23:8082/project/f6dd32c9-bf9c-4660-b7f6-0fc3933b4d03 2025-07-23 18:14:52,849 - DEBUG - Requesting: http://10.0.0.23:8082/virtual-network/95c43151-b699-4ed8-9703-ff7b7f142752 2025-07-23 18:14:52,861 - DEBUG - =>VN ctest-vn-25477738 has no policy to be verified 2025-07-23 18:14:52,861 - DEBUG - Verifying the vn in opserver 2025-07-23 18:14:52,861 - DEBUG - Verifying the default-domain:ctest-FatFlowSerialIpv6-31491632:ctest-vn-25477738 virtual network link through opserver 10.0.0.23 2025-07-23 18:14:52,861 - DEBUG - Requesting: http://10.0.0.23:8081/analytics/uves/virtual-networks 2025-07-23 18:14:53,152 - DEBUG - Requesting: http://10.0.0.23:8081/analytics/uves/virtual-networks 2025-07-23 18:14:53,210 - DEBUG - vn link and name as {'name': 'default-domain:ctest-FatFlowSerialIpv6-31491632:ctest-vn-25477738', 'href': 'http://10.0.0.23:8081/analytics/uves/virtual-network/default-domain:ctest-FatFlowSerialIpv6-31491632:ctest-vn-25477738?flat'} 2025-07-23 18:14:53,210 - DEBUG - Verifying the default-domain:ctest-FatFlowSerialIpv6-31491632:ctest-vn-25477738 virtual network link through opserver 10.0.0.249 2025-07-23 18:14:53,210 - DEBUG - Requesting: http://10.0.0.249:8081/analytics/uves/virtual-networks 2025-07-23 18:14:53,508 - DEBUG - Requesting: http://10.0.0.249:8081/analytics/uves/virtual-networks 2025-07-23 18:14:53,562 - DEBUG - vn link and name as {'name': 'default-domain:ctest-FatFlowSerialIpv6-31491632:ctest-vn-25477738', 'href': 'http://10.0.0.249:8081/analytics/uves/virtual-network/default-domain:ctest-FatFlowSerialIpv6-31491632:ctest-vn-25477738?flat'} 2025-07-23 18:14:53,562 - DEBUG - Verifying the default-domain:ctest-FatFlowSerialIpv6-31491632:ctest-vn-25477738 virtual network link through opserver 10.0.0.73 2025-07-23 18:14:53,562 - DEBUG - Requesting: http://10.0.0.73:8081/analytics/uves/virtual-networks 2025-07-23 18:14:53,859 - DEBUG - Requesting: http://10.0.0.73:8081/analytics/uves/virtual-networks 2025-07-23 18:14:53,912 - DEBUG - vn link and name as {'name': 'default-domain:ctest-FatFlowSerialIpv6-31491632:ctest-vn-25477738', 'href': 'http://10.0.0.73:8081/analytics/uves/virtual-network/default-domain:ctest-FatFlowSerialIpv6-31491632:ctest-vn-25477738?flat'} 2025-07-23 18:14:53,912 - INFO - Validated that VN default-domain:ctest-FatFlowSerialIpv6-31491632:ctest-vn-25477738 is found in opserver 2025-07-23 18:14:53,949 - DEBUG - Do not have enough data to verify VN in agent 2025-07-23 18:14:53,967 - DEBUG - VRF ids for VN ctest-vn-25477738: {} 2025-07-23 18:14:53,988 - DEBUG - Services list from nova: [, , , ] 2025-07-23 18:14:53,988 - DEBUG - Hosts: {'nova': ['an-jenkins-deploy-platform-ansible-os-3704-2', 'an-jenkins-deploy-platform-ansible-os-3704-1']} 2025-07-23 18:14:53,988 - DEBUG - Hosts: {'nova': ['an-jenkins-deploy-platform-ansible-os-3704-2', 'an-jenkins-deploy-platform-ansible-os-3704-1']} 2025-07-23 18:16:34,392 - DEBUG - Installing image ubuntu 2025-07-23 18:16:34,392 - DEBUG - Image info {'name': 'ubuntu.vmdk.gz', 'webserver': '', 'location': '/images/converts/', 'username': 'ubuntu', 'password': 'ubuntu', 'type': 'os', 'flavor': 'contrail_flavor_tiny', 'params': '--container-format bare --disk-format vmdk --property vmware_disktype="sparse" --property vmware_adaptertype="ide" --property hw_scsi_model=virtio-scsi --property hw_disk_bus_model=virtio-scsi --property hw_disk_bus=virtio', 'name_docker': 'phusion-baseimage-enablesshd'} 2025-07-23 18:16:34,393 - DEBUG - Image build path file:///contrail-test/images/ubuntu.vmdk.gz 2025-07-23 18:16:34,393 - DEBUG - Download image from local file file:///contrail-test/images/ubuntu.vmdk.gz 2025-07-23 18:16:34,393 - DEBUG - Unzip image 2025-07-23 18:16:34,393 - DEBUG - Image download (local=True) (proxy=None): Execute cmd: gunzip -f /contrail-test/images/ubuntu.vmdk.gz 2025-07-23 18:16:34,410 - DEBUG - Try to create image in user context 2025-07-23 18:16:34,479 - INFO - Deleting VN ctest-vn-25477738 2025-07-23 18:16:34,676 - DEBUG - Response for deleting network () 2025-07-23 18:16:34,677 - DEBUG - Requesting: http://10.0.0.23:8082/routing-instance/71328e98-349c-445b-8863-2a11c939ca22 2025-07-23 18:16:34,684 - DEBUG - Response Code: 404 2025-07-23 18:16:34,685 - DEBUG - Requesting: http://10.0.0.23:8082/domains 2025-07-23 18:16:34,692 - DEBUG - Requesting: http://10.0.0.23:8082/domain/85afe8be-97e7-4e27-9667-ee7272dde692 2025-07-23 18:16:34,713 - DEBUG - Requesting: http://10.0.0.23:8082/project/f6dd32c9-bf9c-4660-b7f6-0fc3933b4d03 2025-07-23 18:16:34,817 - INFO - Validated that VN ctest-vn-25477738 is not found in API Server 2025-07-23 18:16:34,844 - DEBUG - VN ctest-vn-25477738 is not present in Agent 10.0.0.50 2025-07-23 18:16:34,856 - DEBUG - VN ctest-vn-25477738 is not present in Agent 10.0.0.130 2025-07-23 18:16:34,856 - INFO - Validated that VN ctest-vn-25477738 is not in any agent 2025-07-23 18:16:34,902 - INFO - Validated that Routing instances and Config db in Control-nodes does not have VN ctest-vn-25477738 info 2025-07-23 18:16:36,201 - ERROR -
Traceback (most recent call last):
  File "/contrail-test/tcutils/wrappers.py", line 80, in wrapper
    result = function(self, *args, **kwargs)
  File "/contrail-test/serial_scripts/vrouter/test_fat_flow_serial.py", line 41, in test_fat_flow_intra_vn_inter_node
    node_name=compute_hosts[0])
  File "/contrail-test/common/vrouter/base.py", line 179, in create_vms
    *args, **kwargs
  File "/contrail-test/common/base.py", line 281, in create_vm
    **kwargs)
  File "/contrail-test/common/base.py", line 254, in create_only_vm
    vm_obj.setUp()
  File "/contrail-test/fixtures/vm_test.py", line 188, in setUp
    self.create()
  File "/contrail-test/fixtures/vm_test.py", line 240, in create
    fixed_ips=self.fixed_ips)
  File "/contrail-test/fixtures/openstack.py", line 118, in create_vm
    return self.nova_h.create_vm(vm_name=vm_name, image_name=image_name, vn_ids=vn_ids, **kwargs)
  File "/contrail-test/fixtures/nova_test.py", line 580, in create_vm
    image = self.get_image(image_name=image_name)
  File "/contrail-test/fixtures/nova_test.py", line 199, in get_image
    self._install_image(image_name=image_name)
  File "/contrail-test/fixtures/nova_test.py", line 339, in _install_image
    return self.copy_and_glance(build_path, image_name, params)
  File "/contrail-test/fixtures/nova_test.py", line 410, in copy_and_glance
    image_id = self.glance_h.create_image(generic_image_name, image_path_real, **params)
  File "/contrail-test/fixtures/glance_test.py", line 50, in create_image
    self.upload_image(obj['id'], filename)
  File "/contrail-test/fixtures/glance_test.py", line 54, in upload_image
    self.obj.images.upload(uuid, open(filename, 'rb'))
FileNotFoundError: [Errno 2] No such file or directory: '/contrail-test/images/ubuntu.vmdk'
2025-07-23 18:16:36,201 - DEBUG - Skipping xmpp flap check 2025-07-23 18:16:36,201 - INFO -
2025-07-23 18:16:36,201 - INFO - END TEST : test_fat_flow_intra_vn_inter_node : FAILED[0:01:47]
2025-07-23 18:16:36,201 - INFO - -------------------------------------------------------------------------------- 2025-07-23 18:16:36,943 - INFO - Deleted project: ctest-FatFlowSerialIpv6-31491632, ID : f6dd32c9-bf9c-4660-b7f6-0fc3933b4d03