Building a filter via Expression trees in C#

This article will look at how to build a filter with Expression trees in C#.

It is an academic exercise how to use Expression trees, you would probably use filters just specifying lambda function with LINQ, but the code shows how you can build an Expression incrementally and compile it to a function. If there is a use-case where LINQ does not fit, perhaps some late-binding scenario or where LINQ does not offer an operator, you can use the approach shows in this article, but the article shows simple usage of Expression trees for introducing Expression trees to C# developers wanting to
test them out in more detail.

The sample code below shows the sample code testing out how to build the Expression incrementally using extension method loading some sample data. An important gotcha is to keep sending in same the objectParameter which is the parameter expression used in the lambda function that is built up, this must be the same parameter. Consider some lambda function of an object 'Person' where the parameter 'x' like:

x => x.Age > 3 && x.Age < 9

The point is that the ParameterExpression x must be the same object, or else we get an error.

FilterHelper.cs

public static class FilterHelper {

	public enum ComparisonOperator {
		Equal,
		LessThan,
		LessThanOrEqual,
		GreaterThan,
		GreaterThanOrEqual,
		NotEqual		
	}
	
	public static Func<TClass, bool> CompileFilter<TClass>(this Expression expression, ParameterExpression objectParameter){
		var expr = Expression.Lambda<Func<TClass, bool>>(expression, false, new List<ParameterExpression>{ objectParameter });
		return expr.Compile();
	}
	
	public static Expression BuildFilter<TClass, TProp>(this Expression previousExpression, Expression<Func<TClass, TProp>> prop, object value,
		ComparisonOperator op, ParameterExpression objectParameter){
		
		var propertyInfo = GetPropertyInfo(prop);
		var propertyToCall = Expression.Property(objectParameter, propertyInfo);		
		var valueToTest = Expression.Constant(value);
		
		Expression operatorExpression = null;
		switch (op)
		{
			case ComparisonOperator.Equal:
				operatorExpression = Expression.Equal(propertyToCall, valueToTest);
				break;
			case ComparisonOperator.NotEqual:
				operatorExpression = Expression.NotEqual(propertyToCall, valueToTest);
				break;
			case ComparisonOperator.LessThan:
				operatorExpression = Expression.LessThan(propertyToCall, valueToTest);
				break;
			case ComparisonOperator.LessThanOrEqual:
				operatorExpression = Expression.LessThanOrEqual(propertyToCall, valueToTest);
				break;
			case ComparisonOperator.GreaterThan:
				operatorExpression = Expression.GreaterThan(propertyToCall, valueToTest);
				break;
			case ComparisonOperator.GreaterThanOrEqual:
				operatorExpression = Expression.GreaterThanOrEqual(propertyToCall, valueToTest);
				break;
		}
		
		if (previousExpression == null){
			return operatorExpression;
		}
		else {
			return Expression.AndAlso(previousExpression, operatorExpression);
		}			
	}

	/// <summary>
	/// Gets the corresponding <see cref="PropertyInfo" /> from an <see cref="Expression" />.
	/// </summary>
	/// <param name="property">The expression that selects the property to get info on.</param>
	/// <returns>The property info collected from the expression.</returns>
	/// <exception cref="ArgumentNullException">When <paramref name="property" /> is <c>null</c>.</exception>
	/// <exception cref="ArgumentException">The expression doesn't indicate a valid property."</exception>
	private static PropertyInfo GetPropertyInfo<T, P>(Expression<Func<T, P>> property)
	{
		if (property == null)
		{
			throw new ArgumentNullException(nameof(property));
		}

		if (property.Body is UnaryExpression unaryExp)
		{
			if (unaryExp.Operand is MemberExpression memberExp)
			{
				return (PropertyInfo)memberExp.Member;
			}
		}
		else if (property.Body is MemberExpression memberExp)
		{
			return (PropertyInfo)memberExp.Member;
		}

		throw new ArgumentException($"The expression doesn't indicate a valid property. [ {property} ]");
	}

}

The sample data uses a POCO Employee as entity class: Employee.cs

public class Employee
{
	public int Id { get; set; }
	public string FirstName { get; set; }
	public string LastName { get; set; }
	public string Department { get; set; }
	public string Position { get; set; }
	public decimal Salary { get; set; }
	public DateTime HireDate { get; set; }
	public DateOnly HireDateOnly
	{
		get { return DateOnly.FromDateTime(HireDate); }
	}
}

The sample program loads up the Json data, then it builds the expression with method BuildFilter shown above and then finally calls CompileFilter to build the expression into a Func<TClass, bool> where TClass is the employee type.

Program.cs

void Main()
{
	string json = File.ReadAllText(Path.Combine(@"C:\Users\SomeUser\Documents\LINQPad Queries\SampleData\Employees.json"));
	var employees = JsonSerializer.Deserialize<List<Employee>>(json, 
new JsonSerializerOptions { PropertyNameCaseInsensitive = true });
	//employees.Dump();

	var objectParameter = Expression.Parameter(typeof(Employee));
	Expression currentFilter = null;
	currentFilter = FilterHelper.BuildFilter<Employee, object>(currentFilter, e => e.Department,
 "Engineering", FilterHelper.ComparisonOperator.Equal, objectParameter)
		.BuildFilter<Employee, object>(e => e.Salary, 79000m, 
FilterHelper.ComparisonOperator.GreaterThan, objectParameter);

	Func<Employee, bool> employeeFilter = currentFilter.CompileFilter<Employee>(objectParameter);
	var matchingEmployees = employees.Where(employeeFilter).ToList();
	matchingEmployees.Dump();
}

Sample data json looks like this - an array of employees

Employees.json

[
    {
      "id": 1,
      "firstName": "Alice",
      "lastName": "Johnson",
      "department": "HR",
      "position": "Manager",
      "salary": 60000,
      "hireDate": "2022-03-15"
    },
    {
      "id": 2,
      "firstName": "Bob",
      "lastName": "Smith",
      "department": "Engineering",
      "position": "Software Engineer",
      "salary": 80000,
      "hireDate": "2021-09-10"
    },
    {
      "id": 3,
      "firstName": "Charlie",
      "lastName": "Brown",
      "department": "Finance",
      "position": "Financial Analyst",
      "salary": 70000,
      "hireDate": "2020-05-20"
    },
    {
      "id": 4,
      "firstName": "David",
      "lastName": "Lee",
      "department": "Marketing",
      "position": "Marketing Specialist",
      "salary": 65000,
      "hireDate": "2019-11-05"
    },
    {
      "id": 5,
      "firstName": "Eva",
      "lastName": "Garcia",
      "department": "Sales",
      "position": "Sales Representative",
      "salary": 75000,
      "hireDate": "2018-07-12"
    },
    {
      "id": 6,
      "firstName": "Frank",
      "lastName": "Wang",
      "department": "Engineering",
      "position": "Senior Developer",
      "salary": 95000,
      "hireDate": "2017-02-28"
    },
    {
      "id": 7,
      "firstName": "Grace",
      "lastName": "Miller",
      "department": "HR",
      "position": "Recruiter",
      "salary": 55000,
      "hireDate": "2016-08-18"
    },
    {
      "id": 8,
      "firstName": "Henry",
      "lastName": "Chen",
      "department": "Finance",
      "position": "Financial Manager",
      "salary": 90000,
      "hireDate": "2015-04-03"
    },
    {
      "id": 9,
      "firstName": "Ivy",
      "lastName": "Nguyen",
      "department": "Marketing",
      "position": "Content Writer",
      "salary": 60000,
      "hireDate": "2014-10-22"
    },
    {
      "id": 10,
      "firstName": "Jack",
      "lastName": "Kim",
      "department": "Sales",
      "position": "Account Executive",
      "salary": 80000,
      "hireDate": "2013-06-14"
    },
    {
      "id": 11,
      "firstName": "Karen",
      "lastName": "Taylor",
      "department": "Engineering",
      "position": "QA Engineer",
      "salary": 75000,
      "hireDate": "2012-01-09"
    },
    {
      "id": 12,
      "firstName": "Leo",
      "lastName": "Rodriguez",
      "department": "HR",
      "position": "HR Specialist",
      "salary": 55000,
      "hireDate": "2011-07-27"
    },
    {
      "id": 13,
      "firstName": "Mia",
      "lastName": "Liu",
      "department": "Finance",
      "position": "Financial Advisor",
      "salary": 70000,
      "hireDate": "2010-03-16"
    },
    {
      "id": 14,
      "firstName": "Nina",
      "lastName": "Martinez",
      "department": "Marketing",
      "position": "Social Media Manager",
      "salary": 65000,
      "hireDate": "2009-09-05"
    },
    {
      "id": 15,
      "firstName": "Oscar",
      "lastName": "Hernandez",
      "department": "Sales",
      "position": "Sales Manager",
      "salary": 100000,
      "hireDate": "2008-04-21"
    }
  ]

The filter is more limited than just sticking to LINQ, but the code in this example shows how you can build a filter incrementally. Traditionally, you would use Linq and an IEnumerable of TClass and you can keep on filter it too. Here are some closing arguments for why you could make use of Expression trees and have to use them too and not be able to use Linq:

Purpose: Expression trees represent code as data structures. They allow you to build executable code dynamically in C#.
Use Cases:
- Dynamic Code Generation: When you need to create or modify code at runtime (e.g., building custom queries or transformations).
- Remote Execution: Expression trees are useful for scenarios where you want to send calculations across the wire (e.g., database queries, web services).
- Custom Query Providers: If you’re building your own query provider (like LINQ to SQL or Entity Framework), expression trees help translate queries into other formats (e.g., SQL).

Importing Json File to SQL Server into a variable

A short article today of how to import JSON file to SQL Server into a variable, which can then
be used to insert it into a column of type NVARCHAR(MAX) of a table. The maximum size of NVARCHAR(MAX) is 2 Gb, so you can
import large Json files using this datatype. If the Json is small and below 4000 chars, use for example NVARCHAR(4000) instead. Here is a SQL script to import the json file using OPENROWSET and Bulk import. We also pass in the path to the folder where the json file is. It is put in the same folder as the .sql file script. Note that the variable $(FullScriptDir) is passed in via a .bat file (shown further below) and we expect the .json file to be in the same folder as the .bat file. You can provide a full path to a .json file instead and skip the .bat file here and import a json file, but it is nice to load the .json file from the same folder as the .sql file in case you want to copy the .sql and .json file to another server and not having to provide and possibly having to adjust the full path. Sql-script import_json_file_openrowset.sql:

DECLARE @JSONFILE VARCHAR(MAX); 

SELECT @JSONFILE = BulkColumn
FROM OPENROWSET (BULK '$(FullScriptDir)\top-posts.json', SINGLE_CLOB) AS j;

PRINT 'JsonFile contents: ' + @JSONFILE

IF (ISJSON(@JSONFILE)=1) PRINT 'It is valid Json';

The .bat file here passes the current folder as a variable to the sql script runsqlscript.bat

@set FullScriptDir=%CD%
sqlcmd -S .\SQLEXPRESS  -i import_json_file_openrowset.sql

This outputs:

sqlcmd -S .\SQLEXPRESS  -i import_json_file_openrowset.sql
JsonFile contents: [
   {
      "Id":6107,
      "Score":176,
      "ViewCount":155988,
      "Title":"What are deconvolutional layers?",
      "OwnerUserId":8820
   },
   {
      "Id":155,
      "Score":164,
      "ViewCount":25822,
      "Title":"Publicly Available Datasets",
      "OwnerUserId":227
   }
]
It is valid Json

With the variable JSONFILE you can do whatever with it such as inserting it to a column in a new row of a table for example.
Importing json from a string directly using OPENJSON

It is also possible to directly just import the JSON from a string variable like this:

DECLARE @JSONSTRINGSAMPLE VARCHAR(MAX) 

SET @JSONSTRINGSAMPLE = N'[
 {
    "Id": 2334,
    "Score": 4.3,
    "Title": "Python - Used as scientific tool for graphing"
 },
{
    "Id": 2335,
    "Score": 5.2,
    "Title": "C# : Math and physics programming"
 }
]';

SELECT * FROM OPENJSON (@JSONSTRINGSAMPLE) WITH (
    Id INT,
    Score REAL,
    Title NVARCHAR(100)
)

Functional programming - Fork combinator in C# to combine results from parts

This article will discuss a wellknown combinator called Fork which allows you to combine the mapped result. Consider the following extension methods to fork on an object. Fork here means to operate on parts of the object such as
different properties and apply functions on these parts and then recombine the results into a combined result via a specified combinator function, sometimes called a 'join function'.

public static class FunctionalExtensions {

	public static TOutput Map<TInput, TOutput>(
		this TInput @this,
		Func<TInput, TOutput> func) => func(@this);

	public static TOutput Fork<TInput, TMiddle, TOutput>(
		this TInput @this,
		Func<IEnumerable<TMiddle>, TOutput> combineFunc,
		params Func<TInput, TMiddle>[] parts)
	{
		var intermediateResults = parts.Select(p => p(@this));
		var result = combineFunc(intermediateResults);
		return result;
    }

	public static TOutput Fork<TInput, TMiddle, TOutput>(
		this TInput @this,
		Func<TInput, TMiddle> leftFunc,
		Func<TInput, TMiddle> rightFunc,
		Func<TMiddle, TMiddle, TOutput> combineFunc)
	{
		var leftResult = leftFunc(@this); // @this.Map(leftFunc);
		var rightResult = rightFunc(@this); // @this.Map(rightFunc);
		var combineResult = combineFunc(leftResult, rightResult);
		return combineResult;
	}

}

Let's take a familiar mathematical example, calculating the Hypotenuse in a triangle using Pythagorean theorem. This states that the length of the longest side A of a 'right triangle' is the square root of the sum of the squares of the shorter sides B and C : A = √(B² + C²) Consider this class:

  
  
  public class Triangle {
	public double CathetusA { get; set; }
	public double CathetusB { get; set; }	
	public double Hypotenuse { get; set; }
  }
  
    

Let's test the first Fork helper extension method accepting two functions for specifying the left and right components:

  
  
  	var triangle = new Triangle
	{
		CathetusA = 3,
		CathetusB = 4
	};
	
	triangle.Hypotenuse = triangle.Fork(	
		t => t.CathetusA * t.CathetusA, 
		t => t.CathetusB * t.CathetusB, 
		(l,r) => Math.Sqrt(l+r));
		
	Console.WriteLine(triangle.Hypotenuse);
  
  
  

This yields '5' as the answer via the forked result above. A simple example, but this allows us to create a simple combinatory logic example on an object of any type using functional programming (FP). Let's look at a simpler example just combining multiple properties of an object with a simple string-join, but using the Fork version supporting arbitrary number of parts / components:

 


public class Person {
	public string JobTitle { get; set; }
	public string FirstName { get; set; }
	public IEnumerable<string> MiddleNames { get; set; }
	public string LastName { get; set; }
}

var person = new Person{
		JobTitle = "Detective",
		FirstName = "Alexander",
		MiddleNames = new[] { "James", "Axel" },
		LastName = "Foley"
	};
	
string contactCardText = person.Fork(parts => string.Join(" ", parts), p => p.FirstName,
 p => string.Join(" ", p.MiddleNames), p => p.LastName);
Console.WriteLine(contactCardText);

This yields: Alexander James Axel Foley Fork can be very useful in many cases you need to 'branch off' on an object and recombine parts of the object with some specific function, either two parts or multiple parts and either continue to work on the results or retrieve the results.