2025-07-22 17:11:19,070 - INFO - Domain Default found not creating 2025-07-22 17:11:19,226 - INFO - Project ctest-FatFlowAggr-89876987 not found, creating it 2025-07-22 17:11:19,866 - INFO - Created Project:ctest-FatFlowAggr-89876987, ID : e5124acc-329a-41d2-b356-6a628f9e538e 2025-07-22 17:11:21,311 - INFO - ================================================================================ 2025-07-22 17:11:21,311 - INFO - STARTING TEST : test_fat_flow_aggr_dest_icmp_intra_vn_inter_node 2025-07-22 17:11:21,311 - INFO - TEST DESCRIPTION : : Description: Verify fat flow prefix aggr dest (IPv4) for intra-vn inter-node Steps: 1. Create 1 VN and launch 3 VMs.2 client VMs on same node and server VM on different node. Client 1 in subnet 1, Client 2 in the next subnet. 2. On server VM, config fat flow aggr prefix dest len 25 for ICMP port 0. 3. From both the client VMs, send ICMP traffic to the server VM twice with diff. src ports Pass criteria: 1. On the remote CN, expect 2 pairs ( 1 for client 1, 1 for client 2) of fat flows with prefix aggregated for the src IPs (VM to fabric, Prefix Aggr Dest: Aggregation happens for SRC IPs) 2. On client VM compute nodes, expect 4 pairs of flows and on server compute, expect 2 pairs of flows 3. On server compute node, flow's source port should be 0 for fat flows Maintainer: Ankitja@juniper.net 2025-07-22 17:11:22,588 - DEBUG - Nothing to compare xmpp stats {'10.0.0.19': {'10.20.0.14': '0', '10.20.0.17': '0'}, '10.0.0.27': {'10.20.0.14': '0', '10.20.0.254': '0'}} with 2025-07-22 17:11:22,588 - INFO - Initial checks done. Running the testcase now 2025-07-22 17:11:22,588 - INFO - 2025-07-22 17:11:22,612 - DEBUG - Services list from nova: [, , , ] 2025-07-22 17:11:22,613 - DEBUG - Hosts: {'nova': ['an-jenkins-deploy-platform-ansible-os-3693-2', 'an-jenkins-deploy-platform-ansible-os-3693-1']} 2025-07-22 17:11:23,300 - DEBUG - Response for create_network : {'network': {'id': 'ff236ef0-8515-4de4-b95b-e66ee556f859', 'name': 'ctest-vn-99224963', 'tenant_id': 'e5124acc329a41d2b3566a628f9e538e', 'project_id': 'e5124acc329a41d2b3566a628f9e538e', 'admin_state_up': True, 'shared': False, 'status': 'ACTIVE', 'router:external': False, 'mtu': 0, 'subnets': [], 'fq_name': ['default-domain', 'ctest-FatFlowAggr-89876987', 'ctest-vn-99224963'], 'port_security_enabled': True, 'description': ''}} 2025-07-22 17:11:23,626 - DEBUG - Response for create_subnet : {'subnet': {'id': 'fd2c411f-741b-4854-be6b-6fa86bde0794', 'name': '', 'tenant_id': 'e5124acc329a41d2b3566a628f9e538e', 'network_id': 'ff236ef0-8515-4de4-b95b-e66ee556f859', 'ip_version': 4, 'cidr': '202.212.196.0/26', 'allocation_pools': [{'start': '202.212.196.2', 'end': '202.212.196.62'}], 'gateway_ip': '202.212.196.1', 'enable_dhcp': True, 'ipv6_ra_mode': None, 'ipv6_address_mode': None, 'dns_nameservers': [], 'host_routes': [], 'dns_server_address': '202.212.196.2', 'tags': [], 'project_id': 'e5124acc329a41d2b3566a628f9e538e'}} 2025-07-22 17:11:23,647 - INFO - Created VN ctest-vn-99224963 2025-07-22 17:11:23,660 - DEBUG - VN ctest-vn-99224963 UUID is ff236ef0-8515-4de4-b95b-e66ee556f859 2025-07-22 17:11:23,956 - DEBUG - Requesting: http://10.0.0.254:8082/domains 2025-07-22 17:11:24,260 - DEBUG - Requesting: http://10.0.0.254:8082/domains 2025-07-22 17:11:24,288 - DEBUG - Requesting: http://10.0.0.254:8082/domain/6830e6da-995f-41aa-841e-c47f32c16910 2025-07-22 17:11:24,318 - DEBUG - Requesting: http://10.0.0.254:8082/project/e5124acc-329a-41d2-b356-6a628f9e538e 2025-07-22 17:11:24,424 - DEBUG - Requesting: http://10.0.0.254:8082/virtual-network/ff236ef0-8515-4de4-b95b-e66ee556f859 2025-07-22 17:11:24,438 - DEBUG - Requesting: http://10.0.0.254:8082/virtual-network/ff236ef0-8515-4de4-b95b-e66ee556f859 2025-07-22 17:11:24,449 - DEBUG - Requesting: http://10.0.0.254:8082/routing-instance/587605e7-d3b6-4120-8141-c94cf91cb159 2025-07-22 17:11:24,473 - DEBUG - Requesting: http://10.0.0.254:8082/routing-instance/587605e7-d3b6-4120-8141-c94cf91cb159 2025-07-22 17:11:24,481 - DEBUG - Requesting: http://10.0.0.254:8082/route-target/bc435f9c-ed93-49b0-9a71-dd68e59d3c23 2025-07-22 17:11:24,489 - DEBUG - Requesting: http://10.0.0.254:8082/route-target/f4fd8a6f-56f6-4abe-8e02-8701708d7c04 2025-07-22 17:11:24,497 - DEBUG - Route Targets: ['target:64512:10000', 'target:64512:8000005'] 2025-07-22 17:11:24,497 - DEBUG - Requesting: http://10.0.0.254:8082/virtual-network/ff236ef0-8515-4de4-b95b-e66ee556f859 2025-07-22 17:11:24,510 - DEBUG - Requesting: http://10.0.0.254:8082/routing-instance/587605e7-d3b6-4120-8141-c94cf91cb159 2025-07-22 17:11:24,532 - INFO - Verified VN network id 12 for VN ff236ef0-8515-4de4-b95b-e66ee556f859 2025-07-22 17:11:24,533 - INFO - Verifications in API Server for VN ctest-vn-99224963 passed 2025-07-22 17:11:24,533 - DEBUG - Requesting: http://10.0.0.254:8082/virtual-network/ff236ef0-8515-4de4-b95b-e66ee556f859 2025-07-22 17:11:24,545 - DEBUG - Requesting: http://10.0.0.254:8082/routing-instance/587605e7-d3b6-4120-8141-c94cf91cb159 2025-07-22 17:11:24,556 - DEBUG - Requesting: http://10.0.0.254:8082/routing-instance/587605e7-d3b6-4120-8141-c94cf91cb159 2025-07-22 17:11:24,567 - DEBUG - Requesting: http://10.0.0.254:8082/route-target/bc435f9c-ed93-49b0-9a71-dd68e59d3c23 2025-07-22 17:11:24,577 - DEBUG - Requesting: http://10.0.0.254:8082/route-target/f4fd8a6f-56f6-4abe-8e02-8701708d7c04 2025-07-22 17:11:24,597 - DEBUG - Control-node 10.0.0.254 : VN object is : {'node_name': 'virtual-network:default-domain:ctest-FatFlowAggr-89876987:ctest-vn-99224963', 'interests': '-', 'advertised': '-', 'dbentryflags': None, 'obj_info': {'sequence_number': '0', 'origin': 'Cassandra', 'data': {'id-perms': {'permissions': {'owner': 'ctest-FatFlowAggr-89876987', 'owner-access': '7', 'group': 'reader', 'group-access': '7', 'other-access': '7'}, 'uuid': {'uuid-mslong': '18384660083160468964', 'uuid-lslong': '13356522483865811033'}, 'enable': 'true', 'created': '2025-07-22T17:11:23', 'last-modified': '2025-07-22T17:11:23', 'description': None, 'user-visible': 'true', 'creator': None}}}, 'neighbors': None, 'last_modified': '00:00:00.785696'} 2025-07-22 17:11:24,604 - DEBUG - Route Targets: ['target:64512:10000', 'target:64512:8000005'] 2025-07-22 17:11:24,612 - DEBUG - Control-node 10.0.0.50 : VN object is : {'node_name': 'virtual-network:default-domain:ctest-FatFlowAggr-89876987:ctest-vn-99224963', 'interests': '-', 'advertised': '-', 'dbentryflags': None, 'obj_info': {'sequence_number': '0', 'origin': 'Cassandra', 'data': {'id-perms': {'permissions': {'owner': 'ctest-FatFlowAggr-89876987', 'owner-access': '7', 'group': 'reader', 'group-access': '7', 'other-access': '7'}, 'uuid': {'uuid-mslong': '18384660083160468964', 'uuid-lslong': '13356522483865811033'}, 'enable': 'true', 'created': '2025-07-22T17:11:23', 'last-modified': '2025-07-22T17:11:23', 'description': None, 'user-visible': 'true', 'creator': None}}}, 'neighbors': None, 'last_modified': '00:00:00.809118'} 2025-07-22 17:11:24,620 - DEBUG - Route Targets: ['target:64512:10000', 'target:64512:8000005'] 2025-07-22 17:11:24,626 - DEBUG - Control-node 10.0.0.23 : VN object is : {'node_name': 'virtual-network:default-domain:ctest-FatFlowAggr-89876987:ctest-vn-99224963', 'interests': '-', 'advertised': '-', 'dbentryflags': None, 'obj_info': {'sequence_number': '0', 'origin': 'Cassandra', 'data': {'id-perms': {'permissions': {'owner': 'ctest-FatFlowAggr-89876987', 'owner-access': '7', 'group': 'reader', 'group-access': '7', 'other-access': '7'}, 'uuid': {'uuid-mslong': '18384660083160468964', 'uuid-lslong': '13356522483865811033'}, 'enable': 'true', 'created': '2025-07-22T17:11:23', 'last-modified': '2025-07-22T17:11:23', 'description': None, 'user-visible': 'true', 'creator': None}}}, 'neighbors': None, 'last_modified': '00:00:00.825199'} 2025-07-22 17:11:24,635 - DEBUG - Route Targets: ['target:64512:10000', 'target:64512:8000005'] 2025-07-22 17:11:24,635 - INFO - On all control nodes, Config, RI and RT verification for VN ctest-vn-99224963 passed 2025-07-22 17:11:24,636 - DEBUG - ====Verifying policy data for ctest-vn-99224963 in API_Server ====== 2025-07-22 17:11:24,636 - DEBUG - Requesting: http://10.0.0.254:8082/domains 2025-07-22 17:11:24,642 - DEBUG - Requesting: http://10.0.0.254:8082/domain/6830e6da-995f-41aa-841e-c47f32c16910 2025-07-22 17:11:24,656 - DEBUG - Requesting: http://10.0.0.254:8082/project/e5124acc-329a-41d2-b356-6a628f9e538e 2025-07-22 17:11:24,758 - DEBUG - Requesting: http://10.0.0.254:8082/virtual-network/ff236ef0-8515-4de4-b95b-e66ee556f859 2025-07-22 17:11:24,771 - DEBUG - =>VN ctest-vn-99224963 has no policy to be verified 2025-07-22 17:11:24,771 - DEBUG - Verifying the vn in opserver 2025-07-22 17:11:24,771 - DEBUG - Verifying the default-domain:ctest-FatFlowAggr-89876987:ctest-vn-99224963 virtual network link through opserver 10.0.0.254 2025-07-22 17:11:24,772 - DEBUG - Requesting: http://10.0.0.254:8081/analytics/uves/virtual-networks 2025-07-22 17:11:25,075 - DEBUG - Requesting: http://10.0.0.254:8081/analytics/uves/virtual-networks 2025-07-22 17:11:25,151 - DEBUG - vn link and name as {'name': 'default-domain:ctest-FatFlowAggr-89876987:ctest-vn-99224963', 'href': 'http://10.0.0.254:8081/analytics/uves/virtual-network/default-domain:ctest-FatFlowAggr-89876987:ctest-vn-99224963?flat'} 2025-07-22 17:11:25,151 - DEBUG - Verifying the default-domain:ctest-FatFlowAggr-89876987:ctest-vn-99224963 virtual network link through opserver 10.0.0.50 2025-07-22 17:11:25,152 - DEBUG - Requesting: http://10.0.0.50:8081/analytics/uves/virtual-networks 2025-07-22 17:11:25,453 - DEBUG - Requesting: http://10.0.0.50:8081/analytics/uves/virtual-networks 2025-07-22 17:11:25,530 - DEBUG - vn link and name as {'name': 'default-domain:ctest-FatFlowAggr-89876987:ctest-vn-99224963', 'href': 'http://10.0.0.50:8081/analytics/uves/virtual-network/default-domain:ctest-FatFlowAggr-89876987:ctest-vn-99224963?flat'} 2025-07-22 17:11:25,530 - DEBUG - Verifying the default-domain:ctest-FatFlowAggr-89876987:ctest-vn-99224963 virtual network link through opserver 10.0.0.23 2025-07-22 17:11:25,531 - DEBUG - Requesting: http://10.0.0.23:8081/analytics/uves/virtual-networks 2025-07-22 17:11:25,826 - DEBUG - Requesting: http://10.0.0.23:8081/analytics/uves/virtual-networks 2025-07-22 17:11:25,885 - DEBUG - vn link and name as {'name': 'default-domain:ctest-FatFlowAggr-89876987:ctest-vn-99224963', 'href': 'http://10.0.0.23:8081/analytics/uves/virtual-network/default-domain:ctest-FatFlowAggr-89876987:ctest-vn-99224963?flat'} 2025-07-22 17:11:25,885 - INFO - Validated that VN default-domain:ctest-FatFlowAggr-89876987:ctest-vn-99224963 is found in opserver 2025-07-22 17:11:25,911 - DEBUG - Do not have enough data to verify VN in agent 2025-07-22 17:11:25,924 - DEBUG - VRF ids for VN ctest-vn-99224963: {} 2025-07-22 17:11:25,977 - DEBUG - Hosts: {'nova': ['an-jenkins-deploy-platform-ansible-os-3693-2', 'an-jenkins-deploy-platform-ansible-os-3693-1']} 2025-07-22 17:13:06,353 - DEBUG - Installing image ubuntu 2025-07-22 17:13:06,353 - DEBUG - Image info {'name': 'ubuntu.vmdk.gz', 'webserver': '', 'location': '/images/converts/', 'username': 'ubuntu', 'password': 'ubuntu', 'type': 'os', 'flavor': 'contrail_flavor_tiny', 'params': '--container-format bare --disk-format vmdk --property vmware_disktype="sparse" --property vmware_adaptertype="ide" --property hw_scsi_model=virtio-scsi --property hw_disk_bus_model=virtio-scsi --property hw_disk_bus=virtio', 'name_docker': 'phusion-baseimage-enablesshd'} 2025-07-22 17:13:06,353 - DEBUG - Image build path file:///contrail-test/images/ubuntu.vmdk.gz 2025-07-22 17:13:06,353 - DEBUG - Download image from local file file:///contrail-test/images/ubuntu.vmdk.gz 2025-07-22 17:13:06,353 - DEBUG - Unzip image 2025-07-22 17:13:06,353 - DEBUG - Image download (local=True) (proxy=None): Execute cmd: gunzip -f /contrail-test/images/ubuntu.vmdk.gz 2025-07-22 17:13:06,378 - DEBUG - Try to create image in user context 2025-07-22 17:13:06,451 - DEBUG - Deleting RT for VN ctest-vn-99224963 2025-07-22 17:13:06,576 - INFO - Deleting VN ctest-vn-99224963 2025-07-22 17:13:06,830 - DEBUG - Response for deleting network () 2025-07-22 17:13:06,830 - DEBUG - Requesting: http://10.0.0.254:8082/routing-instance/587605e7-d3b6-4120-8141-c94cf91cb159 2025-07-22 17:13:06,838 - DEBUG - Response Code: 404 2025-07-22 17:13:06,838 - DEBUG - Requesting: http://10.0.0.254:8082/domains 2025-07-22 17:13:06,845 - DEBUG - Requesting: http://10.0.0.254:8082/domain/6830e6da-995f-41aa-841e-c47f32c16910 2025-07-22 17:13:06,862 - DEBUG - Requesting: http://10.0.0.254:8082/project/e5124acc-329a-41d2-b356-6a628f9e538e 2025-07-22 17:13:06,972 - INFO - Validated that VN ctest-vn-99224963 is not found in API Server 2025-07-22 17:13:06,989 - DEBUG - VN ctest-vn-99224963 is not present in Agent 10.0.0.19 2025-07-22 17:13:07,003 - DEBUG - VN ctest-vn-99224963 is not present in Agent 10.0.0.27 2025-07-22 17:13:07,003 - INFO - Validated that VN ctest-vn-99224963 is not in any agent 2025-07-22 17:13:07,049 - INFO - Validated that Routing instances and Config db in Control-nodes does not have VN ctest-vn-99224963 info 2025-07-22 17:13:08,332 - ERROR - 
Traceback (most recent call last):
  File "/contrail-test/tcutils/wrappers.py", line 80, in wrapper
    result = function(self, *args, **kwargs)
  File "/contrail-test/scripts/vrouter/test_fat_flow_aggr.py", line 61, in test_fat_flow_aggr_dest_icmp_intra_vn_inter_node
    dual=ipv6, prefix_length6=prefix_length6, only_v6=only_v6)
  File "/contrail-test/common/vrouter/base.py", line 383, in fat_flow_with_prefix_aggr
    node_name=compute_hosts[0])
  File "/contrail-test/common/vrouter/base.py", line 179, in create_vms
    *args, **kwargs
  File "/contrail-test/common/base.py", line 281, in create_vm
    **kwargs)
  File "/contrail-test/common/base.py", line 254, in create_only_vm
    vm_obj.setUp()
  File "/contrail-test/fixtures/vm_test.py", line 188, in setUp
    self.create()
  File "/contrail-test/fixtures/vm_test.py", line 240, in create
    fixed_ips=self.fixed_ips)
  File "/contrail-test/fixtures/openstack.py", line 118, in create_vm
    return self.nova_h.create_vm(vm_name=vm_name, image_name=image_name, vn_ids=vn_ids, **kwargs)
  File "/contrail-test/fixtures/nova_test.py", line 577, in create_vm
    image = self.get_image(image_name=image_name)
  File "/contrail-test/fixtures/nova_test.py", line 198, in get_image
    self._install_image(image_name=image_name)
  File "/contrail-test/fixtures/nova_test.py", line 338, in _install_image
    return self.copy_and_glance(build_path, image_name, params)
  File "/contrail-test/fixtures/nova_test.py", line 407, in copy_and_glance
    image_id = self.glance_h.create_image(generic_image_name, image_path_real, **params)
  File "/contrail-test/fixtures/glance_test.py", line 50, in create_image
    self.upload_image(obj['id'], filename)
  File "/contrail-test/fixtures/glance_test.py", line 54, in upload_image
    self.obj.images.upload(uuid, open(filename, 'rb'))
FileNotFoundError: [Errno 2] No such file or directory: '/contrail-test/images/ubuntu.vmdk'
2025-07-22 17:13:08,352 - DEBUG - No XMPP flaps were noticed during the test 2025-07-22 17:13:08,352 - INFO - 2025-07-22 17:13:08,352 - INFO - END TEST : test_fat_flow_aggr_dest_icmp_intra_vn_inter_node : FAILED[0:01:47] 2025-07-22 17:13:08,353 - INFO - -------------------------------------------------------------------------------- 2025-07-22 17:13:09,139 - INFO - Deleted project: ctest-FatFlowAggr-89876987, ID : e5124acc-329a-41d2-b356-6a628f9e538e