Sneak Preview: The New ‘Collection Filter’ Element

Alex Edelstein

2 years ago

Collection Filter Overview

A new out of the box collection processor element joins the fray! With the new Collection Filter element you will be able to pass in a collection of records and filter it down. But that’s not all. This element has a few superpowers:

Simplified Flows: It allows the removal of loops, enabling simpler flows
Very Large Record Sets: It works with very large sets of records, extending the upward limits of Flow
Enhanced Condition Specification: It allows direct inline formulas with the ability to traverse to parents

1. Simplifying flows by replacing loops

Here’s an example of how filtering is typically done today (on the left) and how it can be done in Spring ’22 (on the right):

Finding records in a collection with loops

Finding records with the new Filter element

2. Much higher processing limits

In the above three element loop example above, there’s a limit of 2,000 element ‘iterations’ or ‘hits’ in a single flow, which effectively caps the number of records you can process to about 700. Go beyond that and you get the ‘ITERATION_LIMIT_EXCEEDED’ error.

With the new Collection Filter, however, you’ll no longer be bound by this limit. We’ve been able to filter collections of upwards of 50,000 records within seconds! Win.

3. Enhanced condition specification

Let’s first look at how conditions are specified. This will be quite familiar. Suppose you want to filter out all of the members of a collection of Accounts that have AnnualRevenue < 100,000. You can do that easily by setting conditions:

However, you can also choose to filter using an inline formula. Take for example an extension of the above scenario where you need to filter a collection of 12,000 accounts with complex criteria not currently supported by ‘Get Records’. Imagine in this fictitious scenario you need to filter by a combination of RecordType, Account Owner criteria, and Parent Account details.

Using the Formula Expression capabilities of the new Filter element you can now do that:

Additional Usage Notes

Filter Collection Saves a New Collection Instead of Overwriting the Existing One

While the ‘Sort Collection’ element overwrites the collection you pass in, the new Filter Collection creates a new one. We did this so you could have the flexibility to create multiple collections from a source collection of records, giving you more flexibility in how you control how your data ‘flows’.

The CurrentItem Variable Persists After Collection Filter is Removed

When the new Filter Collection element is inserted into a flow, a record variable called CurrentItem_FilterElementName is automatically generated and added to the flow to provide a way to use formulas and expressions for filtration.

If the Collection Filter element is deleted from the flow, the CurrentItem variable will be left as a resource in the flow and will need to be deleted manually.

Formula Performance Impact

Note that complex formulas with large amounts of records will impact performance, but by and large it will still be faster than your standard flow looping.

If you can use Conditions instead of a Formula Evaluation to drive your filter logic, you should always use Conditions due to the performance gains. The performance impact is relatively minimal for <500 records but you may start seeing a difference when you exceed 5,000 to 10,000 records.

Advanced Use Case: Avoid ‘Getting’ in Your Loops

For you intermediate to Jedi level Flownatics out there, this one’s for you! One powerful use case of Filter is that you’ll be able to achieve more efficient looping within your Flows. Note the below example could be solved a million ways, we just wanted to show you a creative way to utilize this new element.

Suppose you have a Flow that updates the Sales Price on a Product Line Item based on a discount determined in a set of Product Discount records. Your discount values are stored as records in Salesforce and you already have the discount records in your flow available to you. You just need to apply the discount for that particular product to the Product Line Item.

When the Opportunity is changed to a stage of ‘Proposal/Price Quote’ you want all of the Product Line Items to get the max discount possible applied. For each of the products tied to the opportunity, you do not want to do a a ‘Get’ inside of your loop to retrieve the relevant Product Discount records because you want to be a good ‘Governor Limit Citizen’.

The two big pieces here are the Collection Filter and Collection Sort elements. Here we are ‘Filtering’ the Discount records we grabbed in the beginning and finding discounts that are related to the product in our line item, then we are sorting it by the biggest Discount Percentage and reducing/limiting the collection to 1 record. In this fictitious scenario we had 2 possible discount records, but we only wanted the one with the biggest discount, hence the ‘LIMIT 1’. We then grab the single record stored in the results of the sort by using a quick loop.