Building a filter via Expression trees in C#

This article will look at how to build a filter with Expression trees in C#.

It is an academic exercise how to use Expression trees, you would probably use filters just specifying lambda function with LINQ, but the code shows how you can build an Expression incrementally and compile it to a function. If there is a use-case where LINQ does not fit, perhaps some late-binding scenario or where LINQ does not offer an operator, you can use the approach shows in this article, but the article shows simple usage of Expression trees for introducing Expression trees to C# developers wanting to
test them out in more detail.

The sample code below shows the sample code testing out how to build the Expression incrementally using extension method loading some sample data. An important gotcha is to keep sending in same the objectParameter which is the parameter expression used in the lambda function that is built up, this must be the same parameter. Consider some lambda function of an object 'Person' where the parameter 'x' like:

x => x.Age > 3 && x.Age < 9

The point is that the ParameterExpression x must be the same object, or else we get an error.

FilterHelper.cs

public static class FilterHelper {

	public enum ComparisonOperator {
		Equal,
		LessThan,
		LessThanOrEqual,
		GreaterThan,
		GreaterThanOrEqual,
		NotEqual		
	}
	
	public static Func<TClass, bool> CompileFilter<TClass>(this Expression expression, ParameterExpression objectParameter){
		var expr = Expression.Lambda<Func<TClass, bool>>(expression, false, new List<ParameterExpression>{ objectParameter });
		return expr.Compile();
	}
	
	public static Expression BuildFilter<TClass, TProp>(this Expression previousExpression, Expression<Func<TClass, TProp>> prop, object value,
		ComparisonOperator op, ParameterExpression objectParameter){
		
		var propertyInfo = GetPropertyInfo(prop);
		var propertyToCall = Expression.Property(objectParameter, propertyInfo);		
		var valueToTest = Expression.Constant(value);
		
		Expression operatorExpression = null;
		switch (op)
		{
			case ComparisonOperator.Equal:
				operatorExpression = Expression.Equal(propertyToCall, valueToTest);
				break;
			case ComparisonOperator.NotEqual:
				operatorExpression = Expression.NotEqual(propertyToCall, valueToTest);
				break;
			case ComparisonOperator.LessThan:
				operatorExpression = Expression.LessThan(propertyToCall, valueToTest);
				break;
			case ComparisonOperator.LessThanOrEqual:
				operatorExpression = Expression.LessThanOrEqual(propertyToCall, valueToTest);
				break;
			case ComparisonOperator.GreaterThan:
				operatorExpression = Expression.GreaterThan(propertyToCall, valueToTest);
				break;
			case ComparisonOperator.GreaterThanOrEqual:
				operatorExpression = Expression.GreaterThanOrEqual(propertyToCall, valueToTest);
				break;
		}
		
		if (previousExpression == null){
			return operatorExpression;
		}
		else {
			return Expression.AndAlso(previousExpression, operatorExpression);
		}			
	}

	/// <summary>
	/// Gets the corresponding <see cref="PropertyInfo" /> from an <see cref="Expression" />.
	/// </summary>
	/// <param name="property">The expression that selects the property to get info on.</param>
	/// <returns>The property info collected from the expression.</returns>
	/// <exception cref="ArgumentNullException">When <paramref name="property" /> is <c>null</c>.</exception>
	/// <exception cref="ArgumentException">The expression doesn't indicate a valid property."</exception>
	private static PropertyInfo GetPropertyInfo<T, P>(Expression<Func<T, P>> property)
	{
		if (property == null)
		{
			throw new ArgumentNullException(nameof(property));
		}

		if (property.Body is UnaryExpression unaryExp)
		{
			if (unaryExp.Operand is MemberExpression memberExp)
			{
				return (PropertyInfo)memberExp.Member;
			}
		}
		else if (property.Body is MemberExpression memberExp)
		{
			return (PropertyInfo)memberExp.Member;
		}

		throw new ArgumentException($"The expression doesn't indicate a valid property. [ {property} ]");
	}

}

The sample data uses a POCO Employee as entity class: Employee.cs

public class Employee
{
	public int Id { get; set; }
	public string FirstName { get; set; }
	public string LastName { get; set; }
	public string Department { get; set; }
	public string Position { get; set; }
	public decimal Salary { get; set; }
	public DateTime HireDate { get; set; }
	public DateOnly HireDateOnly
	{
		get { return DateOnly.FromDateTime(HireDate); }
	}
}

The sample program loads up the Json data, then it builds the expression with method BuildFilter shown above and then finally calls CompileFilter to build the expression into a Func<TClass, bool> where TClass is the employee type.

Program.cs

void Main()
{
	string json = File.ReadAllText(Path.Combine(@"C:\Users\SomeUser\Documents\LINQPad Queries\SampleData\Employees.json"));
	var employees = JsonSerializer.Deserialize<List<Employee>>(json, 
new JsonSerializerOptions { PropertyNameCaseInsensitive = true });
	//employees.Dump();

	var objectParameter = Expression.Parameter(typeof(Employee));
	Expression currentFilter = null;
	currentFilter = FilterHelper.BuildFilter<Employee, object>(currentFilter, e => e.Department,
 "Engineering", FilterHelper.ComparisonOperator.Equal, objectParameter)
		.BuildFilter<Employee, object>(e => e.Salary, 79000m, 
FilterHelper.ComparisonOperator.GreaterThan, objectParameter);

	Func<Employee, bool> employeeFilter = currentFilter.CompileFilter<Employee>(objectParameter);
	var matchingEmployees = employees.Where(employeeFilter).ToList();
	matchingEmployees.Dump();
}

Sample data json looks like this - an array of employees

Employees.json

[
    {
      "id": 1,
      "firstName": "Alice",
      "lastName": "Johnson",
      "department": "HR",
      "position": "Manager",
      "salary": 60000,
      "hireDate": "2022-03-15"
    },
    {
      "id": 2,
      "firstName": "Bob",
      "lastName": "Smith",
      "department": "Engineering",
      "position": "Software Engineer",
      "salary": 80000,
      "hireDate": "2021-09-10"
    },
    {
      "id": 3,
      "firstName": "Charlie",
      "lastName": "Brown",
      "department": "Finance",
      "position": "Financial Analyst",
      "salary": 70000,
      "hireDate": "2020-05-20"
    },
    {
      "id": 4,
      "firstName": "David",
      "lastName": "Lee",
      "department": "Marketing",
      "position": "Marketing Specialist",
      "salary": 65000,
      "hireDate": "2019-11-05"
    },
    {
      "id": 5,
      "firstName": "Eva",
      "lastName": "Garcia",
      "department": "Sales",
      "position": "Sales Representative",
      "salary": 75000,
      "hireDate": "2018-07-12"
    },
    {
      "id": 6,
      "firstName": "Frank",
      "lastName": "Wang",
      "department": "Engineering",
      "position": "Senior Developer",
      "salary": 95000,
      "hireDate": "2017-02-28"
    },
    {
      "id": 7,
      "firstName": "Grace",
      "lastName": "Miller",
      "department": "HR",
      "position": "Recruiter",
      "salary": 55000,
      "hireDate": "2016-08-18"
    },
    {
      "id": 8,
      "firstName": "Henry",
      "lastName": "Chen",
      "department": "Finance",
      "position": "Financial Manager",
      "salary": 90000,
      "hireDate": "2015-04-03"
    },
    {
      "id": 9,
      "firstName": "Ivy",
      "lastName": "Nguyen",
      "department": "Marketing",
      "position": "Content Writer",
      "salary": 60000,
      "hireDate": "2014-10-22"
    },
    {
      "id": 10,
      "firstName": "Jack",
      "lastName": "Kim",
      "department": "Sales",
      "position": "Account Executive",
      "salary": 80000,
      "hireDate": "2013-06-14"
    },
    {
      "id": 11,
      "firstName": "Karen",
      "lastName": "Taylor",
      "department": "Engineering",
      "position": "QA Engineer",
      "salary": 75000,
      "hireDate": "2012-01-09"
    },
    {
      "id": 12,
      "firstName": "Leo",
      "lastName": "Rodriguez",
      "department": "HR",
      "position": "HR Specialist",
      "salary": 55000,
      "hireDate": "2011-07-27"
    },
    {
      "id": 13,
      "firstName": "Mia",
      "lastName": "Liu",
      "department": "Finance",
      "position": "Financial Advisor",
      "salary": 70000,
      "hireDate": "2010-03-16"
    },
    {
      "id": 14,
      "firstName": "Nina",
      "lastName": "Martinez",
      "department": "Marketing",
      "position": "Social Media Manager",
      "salary": 65000,
      "hireDate": "2009-09-05"
    },
    {
      "id": 15,
      "firstName": "Oscar",
      "lastName": "Hernandez",
      "department": "Sales",
      "position": "Sales Manager",
      "salary": 100000,
      "hireDate": "2008-04-21"
    }
  ]

The filter is more limited than just sticking to LINQ, but the code in this example shows how you can build a filter incrementally. Traditionally, you would use Linq and an IEnumerable of TClass and you can keep on filter it too. Here are some closing arguments for why you could make use of Expression trees and have to use them too and not be able to use Linq:

Purpose: Expression trees represent code as data structures. They allow you to build executable code dynamically in C#.
Use Cases:
- Dynamic Code Generation: When you need to create or modify code at runtime (e.g., building custom queries or transformations).
- Remote Execution: Expression trees are useful for scenarios where you want to send calculations across the wire (e.g., database queries, web services).
- Custom Query Providers: If you’re building your own query provider (like LINQ to SQL or Entity Framework), expression trees help translate queries into other formats (e.g., SQL).

Using expression trees to build up loops - Gauss Summation

I tested out Expression trees today in more depth and played around with Gauss summation. The Gauss summation is a well-known theorem in Calculus. It states that for a plain arithmetic sequence of numbers with a distance of 1 (i.e. 1,2,3,4,5,6...) from 1..n the sum of these numbers are equal to the formula : Sum(n) = (n*(n+1)) / 2 Johan Karl Friendrich Gauss is renowned to have come up with this as a very young student when a school teacher asked the class to sum the numbers from 1 to 100 and give him the answer. Gauss almost instantly replied '5050', which was also the correct answer. This may or may not have been the case. The formula itself can anyways be theorized by summing the largest and
smallest number and then approaching the middle of the sequence. You can add 1 and 100 to get 101, 2 and 99 to get 101 and so on. The sum is always 101 (n+1) and there are a hundred such 'pairs' (n). But we want to only sum the numbers once, so we divide by 2 => we have the Gauss summation formula ! Let's look at how to do such a summation using Expression trees in C#. But I have only created a loop algorithm here, we calculate the same answer but we instead use expression trees. In demonstrates how
we can get started with expression trees in C# using loops (it is while loop which is created here) and parameter expressions and other components, such as 'labels' used in 'gotos'. This is actually needed in
expression trees to get the looping and breaking to work. The 'SumRange' method looks like this :

public static Expression SumRange(ParameterExpression value)
{
    LabelTarget label = Expression.Label(typeof(int));

    ParameterExpression result = Expression.Variable(typeof(int), "result");
    var initializeResult = Expression.Assign(result, Expression.Constant(0));

    var innerLogicBlock = Expression.Block(
        Expression.Assign(result,
            Expression.Add(result, value)),
        Expression.PostDecrementAssign(value)
    );

    BlockExpression body = Expression.Block(
       new[] { result },
       initializeResult,
       Expression.Loop(
           Expression.IfThenElse(
            Expression.GreaterThanOrEqual(value, Expression.Constant(1)),
            innerLogicBlock,
            Expression.Break(label, result)
            ),
            label
         )
    );
    return body;
}

We pass in a parameter expression. We then declare a 'label' which is used in a 'goto' execution flow when we want to break out of our loop, created by Expression.Loop. The initializeResult is listed here inside Expression block as we want to assign the result variable to the initial value (the expression constant '0'). We then have an 'outer logic' where we have a If-Then-Else condition where we check if value is greater than or equal to 1 and then
we perform the 'inner logicl block' assigned earlier, where we assign result to itself and the value variable passed in as a parameterexpression to this method. Note, we will do some type checking via Expression.Lambda which call this SumRange method explained further below. Note the use of 'PostDecrementAssign' expression which decrements the 'value' and ensures we can exit out of the loop. It can be of course hard to follow along such expression trees without some tooling. I use the ReadableExpressions:Visualizer plugin for VS 2022 here : https://marketplace.visualstudio.com/items?itemName=vs-publisher-1232914.ReadableExpressionsVisualizers You can use it to preview expressions as shown in the below screen shot :

And our unit test passes with expected result.

 

        [Fact]
        public void SumRange()
        {
            var value = Expression.Parameter(typeof(int));
            var result = ScriptingEngine.SumRange(value);
            var expr = Expression.Lambda<Func<int, int>>(result, value);
            var func = expr.Compile(); 
             Assert.Equal(5050, func(100)); 
        }

 

As you can see, even for a simple method, we need to type a lot of code to build up an expression tree. There are helper libraries such as AgileObjects.ReadableExpressions and System.Linq.Dynamic.Core which
can help out a lot when using expression trees. Add these to your package references (.csproj) for example :

  <ItemGroup>
    <PackageReference Include="AgileObjects.ReadableExpressions" Version="3.3.0" />
    <PackageReference Include="System.Linq.Dynamic.Core" Version="1.2.18" />
  </ItemGroup>

The first package of these got a handy method ToReadableString and the last one got a handy helper class called DynamicExpressionParser, which in tandem can create pretty complex expression trees. When will you use such logic ? Most often when wanting to build up custom logic query filters. You should not allow end users to arbitrarily build up all kinds of query filters, but may offer them a set of user controls to build and combine query filters so they can retrieve data via rather complex rules. The code libs mentioned here is supported in .NET Framework 3.5 or later (and .NET Standard 1.0), so most target frameworks are supported then.