2025-07-24 04:49:18,458 - INFO - Domain Default found not creating 2025-07-24 04:49:18,635 - INFO - Project ctest-FatFlowAggr-79866150 not found, creating it 2025-07-24 04:49:19,292 - INFO - Created Project:ctest-FatFlowAggr-79866150, ID : 626494c0-0c51-4f65-853d-a3764ec08f33 2025-07-24 04:49:20,942 - INFO - ================================================================================ 2025-07-24 04:49:20,943 - INFO - STARTING TEST : test_fat_flow_aggr_dest_icmp_intra_vn_inter_node 2025-07-24 04:49:20,943 - INFO - TEST DESCRIPTION : : Description: Verify fat flow prefix aggr dest (IPv4) for intra-vn inter-node Steps: 1. Create 1 VN and launch 3 VMs.2 client VMs on same node and server VM on different node. Client 1 in subnet 1, Client 2 in the next subnet. 2. On server VM, config fat flow aggr prefix dest len 25 for ICMP port 0. 3. From both the client VMs, send ICMP traffic to the server VM twice with diff. src ports Pass criteria: 1. On the remote CN, expect 2 pairs ( 1 for client 1, 1 for client 2) of fat flows with prefix aggregated for the src IPs (VM to fabric, Prefix Aggr Dest: Aggregation happens for SRC IPs) 2. On client VM compute nodes, expect 4 pairs of flows and on server compute, expect 2 pairs of flows 3. On server compute node, flow's source port should be 0 for fat flows Maintainer: Ankitja@juniper.net 2025-07-24 04:49:22,267 - DEBUG - Nothing to compare xmpp stats {'10.0.0.57': {'10.20.0.17': '0', '10.20.0.129': '0'}, '10.0.0.23': {'10.20.0.129': '0', '10.20.0.17': '0'}} with 2025-07-24 04:49:22,267 - INFO - Initial checks done. Running the testcase now 2025-07-24 04:49:22,267 - INFO - 2025-07-24 04:49:22,304 - DEBUG - Services list from nova: [, , , ] 2025-07-24 04:49:22,304 - DEBUG - Hosts: {'nova': ['an-jenkins-deploy-platform-ansible-os-3712-1', 'an-jenkins-deploy-platform-ansible-os-3712-2']} 2025-07-24 04:49:23,084 - DEBUG - Response for create_network : {'network': {'id': '612efd1a-87b2-40af-9736-cf6f3ba0f9d7', 'name': 'ctest-vn-19220449', 'tenant_id': '626494c00c514f65853da3764ec08f33', 'project_id': '626494c00c514f65853da3764ec08f33', 'admin_state_up': True, 'shared': False, 'status': 'ACTIVE', 'router:external': False, 'mtu': 0, 'subnets': [], 'fq_name': ['default-domain', 'ctest-FatFlowAggr-79866150', 'ctest-vn-19220449'], 'port_security_enabled': True, 'description': ''}} 2025-07-24 04:49:23,418 - DEBUG - Response for create_subnet : {'subnet': {'id': 'd7e8a5eb-b1d9-42be-ab96-d897f99b5e74', 'name': '', 'tenant_id': '626494c00c514f65853da3764ec08f33', 'network_id': '612efd1a-87b2-40af-9736-cf6f3ba0f9d7', 'ip_version': 4, 'cidr': '103.153.226.128/26', 'allocation_pools': [{'start': '103.153.226.130', 'end': '103.153.226.190'}], 'gateway_ip': '103.153.226.129', 'enable_dhcp': True, 'ipv6_ra_mode': None, 'ipv6_address_mode': None, 'dns_nameservers': [], 'host_routes': [], 'dns_server_address': '103.153.226.130', 'tags': [], 'project_id': '626494c00c514f65853da3764ec08f33'}} 2025-07-24 04:49:23,447 - INFO - Created VN ctest-vn-19220449 2025-07-24 04:49:23,464 - DEBUG - VN ctest-vn-19220449 UUID is 612efd1a-87b2-40af-9736-cf6f3ba0f9d7 2025-07-24 04:49:23,772 - DEBUG - Requesting: http://10.0.0.19:8082/domains 2025-07-24 04:49:24,090 - DEBUG - Requesting: http://10.0.0.19:8082/domains 2025-07-24 04:49:24,125 - DEBUG - Requesting: http://10.0.0.19:8082/domain/ef05d1d9-2fbc-48d4-8924-0cbd930e68f2 2025-07-24 04:49:24,159 - DEBUG - Requesting: http://10.0.0.19:8082/project/626494c0-0c51-4f65-853d-a3764ec08f33 2025-07-24 04:49:24,278 - DEBUG - Requesting: http://10.0.0.19:8082/virtual-network/612efd1a-87b2-40af-9736-cf6f3ba0f9d7 2025-07-24 04:49:24,295 - DEBUG - Requesting: http://10.0.0.19:8082/virtual-network/612efd1a-87b2-40af-9736-cf6f3ba0f9d7 2025-07-24 04:49:24,312 - DEBUG - Requesting: http://10.0.0.19:8082/routing-instance/49c0ff0e-eeb3-4b75-9770-a995e2253dd3 2025-07-24 04:49:24,327 - DEBUG - Requesting: http://10.0.0.19:8082/routing-instance/49c0ff0e-eeb3-4b75-9770-a995e2253dd3 2025-07-24 04:49:24,339 - DEBUG - Requesting: http://10.0.0.19:8082/route-target/7568bae1-551e-402b-b6d3-d407ca707eef 2025-07-24 04:49:24,353 - DEBUG - Requesting: http://10.0.0.19:8082/route-target/799d5826-e3ac-46b0-9646-2da18fc897d9 2025-07-24 04:49:24,366 - DEBUG - Route Targets: ['target:64512:8000005', 'target:64512:10000'] 2025-07-24 04:49:24,366 - DEBUG - Requesting: http://10.0.0.19:8082/virtual-network/612efd1a-87b2-40af-9736-cf6f3ba0f9d7 2025-07-24 04:49:24,383 - DEBUG - Requesting: http://10.0.0.19:8082/routing-instance/49c0ff0e-eeb3-4b75-9770-a995e2253dd3 2025-07-24 04:49:24,416 - INFO - Verified VN network id 11 for VN 612efd1a-87b2-40af-9736-cf6f3ba0f9d7 2025-07-24 04:49:24,417 - INFO - Verifications in API Server for VN ctest-vn-19220449 passed 2025-07-24 04:49:24,418 - DEBUG - Requesting: http://10.0.0.19:8082/virtual-network/612efd1a-87b2-40af-9736-cf6f3ba0f9d7 2025-07-24 04:49:24,433 - DEBUG - Requesting: http://10.0.0.19:8082/routing-instance/49c0ff0e-eeb3-4b75-9770-a995e2253dd3 2025-07-24 04:49:24,451 - DEBUG - Requesting: http://10.0.0.19:8082/routing-instance/49c0ff0e-eeb3-4b75-9770-a995e2253dd3 2025-07-24 04:49:24,472 - DEBUG - Requesting: http://10.0.0.19:8082/route-target/7568bae1-551e-402b-b6d3-d407ca707eef 2025-07-24 04:49:24,489 - DEBUG - Requesting: http://10.0.0.19:8082/route-target/799d5826-e3ac-46b0-9646-2da18fc897d9 2025-07-24 04:49:24,514 - DEBUG - Control-node 10.0.0.19 : VN object is : {'node_name': 'virtual-network:default-domain:ctest-FatFlowAggr-79866150:ctest-vn-19220449', 'interests': '-', 'advertised': '-', 'dbentryflags': None, 'obj_info': {'sequence_number': '0', 'origin': 'Cassandra', 'data': {'id-perms': {'permissions': {'owner': 'ctest-FatFlowAggr-79866150', 'owner-access': '7', 'group': 'admin', 'group-access': '7', 'other-access': '7'}, 'uuid': {'uuid-mslong': '7002812760995283119', 'uuid-lslong': '10896124425118218711'}, 'enable': 'true', 'created': '2025-07-24T04:49:22', 'last-modified': '2025-07-24T04:49:23', 'description': None, 'user-visible': 'true', 'creator': None}}}, 'neighbors': None, 'last_modified': '00:00:00.898787'} 2025-07-24 04:49:24,524 - DEBUG - Route Targets: ['target:64512:8000005', 'target:64512:10000'] 2025-07-24 04:49:24,533 - DEBUG - Control-node 10.0.0.38 : VN object is : {'node_name': 'virtual-network:default-domain:ctest-FatFlowAggr-79866150:ctest-vn-19220449', 'interests': '-', 'advertised': '-', 'dbentryflags': None, 'obj_info': {'sequence_number': '0', 'origin': 'Cassandra', 'data': {'id-perms': {'permissions': {'owner': 'ctest-FatFlowAggr-79866150', 'owner-access': '7', 'group': 'admin', 'group-access': '7', 'other-access': '7'}, 'uuid': {'uuid-mslong': '7002812760995283119', 'uuid-lslong': '10896124425118218711'}, 'enable': 'true', 'created': '2025-07-24T04:49:22', 'last-modified': '2025-07-24T04:49:23', 'description': None, 'user-visible': 'true', 'creator': None}}}, 'neighbors': None, 'last_modified': '00:00:00.928029'} 2025-07-24 04:49:24,541 - DEBUG - Route Targets: ['target:64512:8000005', 'target:64512:10000'] 2025-07-24 04:49:24,549 - DEBUG - Control-node 10.0.0.41 : VN object is : {'node_name': 'virtual-network:default-domain:ctest-FatFlowAggr-79866150:ctest-vn-19220449', 'interests': '-', 'advertised': '-', 'dbentryflags': None, 'obj_info': {'sequence_number': '0', 'origin': 'Cassandra', 'data': {'id-perms': {'permissions': {'owner': 'ctest-FatFlowAggr-79866150', 'owner-access': '7', 'group': 'admin', 'group-access': '7', 'other-access': '7'}, 'uuid': {'uuid-mslong': '7002812760995283119', 'uuid-lslong': '10896124425118218711'}, 'enable': 'true', 'created': '2025-07-24T04:49:22', 'last-modified': '2025-07-24T04:49:23', 'description': None, 'user-visible': 'true', 'creator': None}}}, 'neighbors': None, 'last_modified': '00:00:00.941478'} 2025-07-24 04:49:24,558 - DEBUG - Route Targets: ['target:64512:8000005', 'target:64512:10000'] 2025-07-24 04:49:24,558 - INFO - On all control nodes, Config, RI and RT verification for VN ctest-vn-19220449 passed 2025-07-24 04:49:24,558 - DEBUG - ====Verifying policy data for ctest-vn-19220449 in API_Server ====== 2025-07-24 04:49:24,559 - DEBUG - Requesting: http://10.0.0.19:8082/domains 2025-07-24 04:49:24,568 - DEBUG - Requesting: http://10.0.0.19:8082/domain/ef05d1d9-2fbc-48d4-8924-0cbd930e68f2 2025-07-24 04:49:24,590 - DEBUG - Requesting: http://10.0.0.19:8082/project/626494c0-0c51-4f65-853d-a3764ec08f33 2025-07-24 04:49:24,717 - DEBUG - Requesting: http://10.0.0.19:8082/virtual-network/612efd1a-87b2-40af-9736-cf6f3ba0f9d7 2025-07-24 04:49:24,730 - DEBUG - =>VN ctest-vn-19220449 has no policy to be verified 2025-07-24 04:49:24,730 - DEBUG - Verifying the vn in opserver 2025-07-24 04:49:24,730 - DEBUG - Verifying the default-domain:ctest-FatFlowAggr-79866150:ctest-vn-19220449 virtual network link through opserver 10.0.0.19 2025-07-24 04:49:24,731 - DEBUG - Requesting: http://10.0.0.19:8081/analytics/uves/virtual-networks 2025-07-24 04:49:25,050 - DEBUG - Requesting: http://10.0.0.19:8081/analytics/uves/virtual-networks 2025-07-24 04:49:25,120 - DEBUG - vn link and name as {'name': 'default-domain:ctest-FatFlowAggr-79866150:ctest-vn-19220449', 'href': 'http://10.0.0.19:8081/analytics/uves/virtual-network/default-domain:ctest-FatFlowAggr-79866150:ctest-vn-19220449?flat'} 2025-07-24 04:49:25,120 - DEBUG - Verifying the default-domain:ctest-FatFlowAggr-79866150:ctest-vn-19220449 virtual network link through opserver 10.0.0.38 2025-07-24 04:49:25,120 - DEBUG - Requesting: http://10.0.0.38:8081/analytics/uves/virtual-networks 2025-07-24 04:49:25,429 - DEBUG - Requesting: http://10.0.0.38:8081/analytics/uves/virtual-networks 2025-07-24 04:49:25,513 - DEBUG - vn link and name as {'name': 'default-domain:ctest-FatFlowAggr-79866150:ctest-vn-19220449', 'href': 'http://10.0.0.38:8081/analytics/uves/virtual-network/default-domain:ctest-FatFlowAggr-79866150:ctest-vn-19220449?flat'} 2025-07-24 04:49:25,513 - DEBUG - Verifying the default-domain:ctest-FatFlowAggr-79866150:ctest-vn-19220449 virtual network link through opserver 10.0.0.41 2025-07-24 04:49:25,513 - DEBUG - Requesting: http://10.0.0.41:8081/analytics/uves/virtual-networks 2025-07-24 04:49:25,833 - DEBUG - Requesting: http://10.0.0.41:8081/analytics/uves/virtual-networks 2025-07-24 04:49:25,916 - DEBUG - vn link and name as {'name': 'default-domain:ctest-FatFlowAggr-79866150:ctest-vn-19220449', 'href': 'http://10.0.0.41:8081/analytics/uves/virtual-network/default-domain:ctest-FatFlowAggr-79866150:ctest-vn-19220449?flat'} 2025-07-24 04:49:25,916 - INFO - Validated that VN default-domain:ctest-FatFlowAggr-79866150:ctest-vn-19220449 is found in opserver 2025-07-24 04:49:25,948 - DEBUG - Do not have enough data to verify VN in agent 2025-07-24 04:49:25,965 - DEBUG - VRF ids for VN ctest-vn-19220449: {} 2025-07-24 04:49:26,020 - DEBUG - Hosts: {'nova': ['an-jenkins-deploy-platform-ansible-os-3712-1', 'an-jenkins-deploy-platform-ansible-os-3712-2']} 2025-07-24 04:51:06,476 - DEBUG - Installing image ubuntu 2025-07-24 04:51:06,476 - DEBUG - Image info {'name': 'ubuntu.vmdk.gz', 'webserver': '', 'location': '/images/converts/', 'username': 'ubuntu', 'password': 'ubuntu', 'type': 'os', 'flavor': 'contrail_flavor_tiny', 'params': '--container-format bare --disk-format vmdk --property vmware_disktype="sparse" --property vmware_adaptertype="ide" --property hw_scsi_model=virtio-scsi --property hw_disk_bus_model=virtio-scsi --property hw_disk_bus=virtio', 'name_docker': 'phusion-baseimage-enablesshd', 'vctmpl': 'ubuntu.vmtx', 'vcname': 'ubuntu-disk1.vmdk'} 2025-07-24 04:51:06,476 - DEBUG - Image build path file:///contrail-test/images/ubuntu.vmdk.gz 2025-07-24 04:51:06,476 - DEBUG - Download image from local file file:///contrail-test/images/ubuntu.vmdk.gz 2025-07-24 04:51:06,476 - DEBUG - Unzip image 2025-07-24 04:51:06,476 - DEBUG - Image download (local=True) (proxy=None): Execute cmd: gunzip -f /contrail-test/images/ubuntu.vmdk.gz 2025-07-24 04:51:06,498 - DEBUG - Try to create image in user context 2025-07-24 04:51:06,588 - DEBUG - Deleting RT for VN ctest-vn-19220449 2025-07-24 04:51:06,736 - INFO - Deleting VN ctest-vn-19220449 2025-07-24 04:51:06,930 - DEBUG - Response for deleting network () 2025-07-24 04:51:06,930 - DEBUG - Requesting: http://10.0.0.19:8082/routing-instance/49c0ff0e-eeb3-4b75-9770-a995e2253dd3 2025-07-24 04:51:06,939 - DEBUG - Response Code: 404 2025-07-24 04:51:06,939 - DEBUG - Requesting: http://10.0.0.19:8082/domains 2025-07-24 04:51:06,947 - DEBUG - Requesting: http://10.0.0.19:8082/domain/ef05d1d9-2fbc-48d4-8924-0cbd930e68f2 2025-07-24 04:51:07,006 - DEBUG - Requesting: http://10.0.0.19:8082/project/626494c0-0c51-4f65-853d-a3764ec08f33 2025-07-24 04:51:07,138 - INFO - Validated that VN ctest-vn-19220449 is not found in API Server 2025-07-24 04:51:07,155 - DEBUG - VN ctest-vn-19220449 is not present in Agent 10.0.0.57 2025-07-24 04:51:07,168 - DEBUG - VN ctest-vn-19220449 is not present in Agent 10.0.0.23 2025-07-24 04:51:07,168 - INFO - Validated that VN ctest-vn-19220449 is not in any agent 2025-07-24 04:51:07,211 - INFO - Validated that Routing instances and Config db in Control-nodes does not have VN ctest-vn-19220449 info 2025-07-24 04:51:08,516 - ERROR - 
Traceback (most recent call last):
  File "/contrail-test/tcutils/wrappers.py", line 80, in wrapper
    result = function(self, *args, **kwargs)
  File "/contrail-test/scripts/vrouter/test_fat_flow_aggr.py", line 61, in test_fat_flow_aggr_dest_icmp_intra_vn_inter_node
    dual=ipv6, prefix_length6=prefix_length6, only_v6=only_v6)
  File "/contrail-test/common/vrouter/base.py", line 383, in fat_flow_with_prefix_aggr
    node_name=compute_hosts[0])
  File "/contrail-test/common/vrouter/base.py", line 179, in create_vms
    *args, **kwargs
  File "/contrail-test/common/base.py", line 281, in create_vm
    **kwargs)
  File "/contrail-test/common/base.py", line 254, in create_only_vm
    vm_obj.setUp()
  File "/contrail-test/fixtures/vm_test.py", line 188, in setUp
    self.create()
  File "/contrail-test/fixtures/vm_test.py", line 240, in create
    fixed_ips=self.fixed_ips)
  File "/contrail-test/fixtures/openstack.py", line 118, in create_vm
    return self.nova_h.create_vm(vm_name=vm_name, image_name=image_name, vn_ids=vn_ids, **kwargs)
  File "/contrail-test/fixtures/nova_test.py", line 577, in create_vm
    image = self.get_image(image_name=image_name)
  File "/contrail-test/fixtures/nova_test.py", line 198, in get_image
    self._install_image(image_name=image_name)
  File "/contrail-test/fixtures/nova_test.py", line 338, in _install_image
    return self.copy_and_glance(build_path, image_name, params)
  File "/contrail-test/fixtures/nova_test.py", line 407, in copy_and_glance
    image_id = self.glance_h.create_image(generic_image_name, image_path_real, **params)
  File "/contrail-test/fixtures/glance_test.py", line 50, in create_image
    self.upload_image(obj['id'], filename)
  File "/contrail-test/fixtures/glance_test.py", line 54, in upload_image
    self.obj.images.upload(uuid, open(filename, 'rb'))
FileNotFoundError: [Errno 2] No such file or directory: '/contrail-test/images/ubuntu.vmdk'
2025-07-24 04:51:08,532 - DEBUG - No XMPP flaps were noticed during the test 2025-07-24 04:51:08,533 - INFO -
2025-07-24 04:51:08,533 - INFO - END TEST : test_fat_flow_aggr_dest_icmp_intra_vn_inter_node : FAILED[0:01:48]
2025-07-24 04:51:08,533 - INFO - -------------------------------------------------------------------------------- 2025-07-24 04:51:09,354 - INFO - Deleted project: ctest-FatFlowAggr-79866150, ID : 626494c0-0c51-4f65-853d-a3764ec08f33