Build dynamic Linq filters (aka. where() predicates)

Photo by ROMAN ODINTSOV on Pexels.com

As you might have noticed recently, I am in a more “meta” programing mood, to be honest, lately I played a lot with Blazor building some wacky things, and more or less I arrived at the following problem, how can we build a dynamic parameter for a list.Where(item => ??) type clause. Why you ask we might need this ? Well there are a lot of reasons / cases where we might need something like this, but in this specific case, I have a Blazor component that adds some strings to a list and based on that, I want to filter another list. Simple right ?

The “Demo” problem

Let’s say we have a list of strings that we want to filter.

Something along the lines of this:

	var list = new List<string>(){
		"a", "ab", "abc", "abcd", "abcde"
	};

Now, let’s say we would want to be able to pick only the items that contain “c”. We would do something like this.

var normalWay = list.Where(e => e.IndexOf("c") > -1);

An this seems quite straight forward, but let’s say we would also need to check that we also need to have “d”:

var normalWay = list.Where(e => e.IndexOf("c") > -1 && e.IndexOf("d") > -1);

And now, let’s say that we need to do this “c” and “d”, or contain “b”.

var normalWay = list.Where(e => (e.IndexOf("c") > -1 && e.IndexOf("d") > -1) || e.IndexOf("b") > -1);

And you can see this is starting to get a bit out of hand.

Now, in the title we had the “Dynamic” word, so let’s see how can we do this dynamically.

So, lets start simple, we will only take the “and” with the ability of adding multiple requirements:

	var containWay = new List<string>() { "c", "d" };
	
	var withContains = list.Where( e => {
		
		var matchedAll = true;
		foreach(var letter in containWay){
			matchedAll = e.IndexOf(letter) > -1 ?  true : false;
			if(!matchedAll) return false;
		}
		
		return matchedAll;
	});

Now, this works for this specific case, but it is way to specific for my taste, and it only has the “And” / “&&” case. We can do an imagination exercise and see how this will easily go bananas complexity and maintainability wise.

The solution – Dynamic filtering using expression trees

Well, we are not going to build expression trees per se, because, well they are way to complicated for this, and I don’t think that is the best idea, but if you don’t know to much about them, the official documentation is a good place to start.

Now, back to our conundrum, as I said, we are going to use function composition to build our own “home made”, expression filters that will allow us to compose our filters.

Now, I am not going to put you the process of discovery, so here is the class I ended up using and we will be going through how it works after.

public class FilterBuilder<T>
{
	private Stack<Func<T, bool>> stack = new();

	private FilterBuilder(Func<T, bool> filter)
	{
		this.stack.Push(filter);
	}

	public static FilterBuilder<T> Create(Func<T, bool> filter)
	{
		return new FilterBuilder<T>(filter);
	}

	public FilterBuilder<T> And(Func<T, bool> filter)
	{

		var q = this.stack.Pop();


		Func<T, bool> result = (item) =>
		{
			return q(item) && filter(item);
		};

		this.stack.Push(result);
		
		return this;

	}

	public FilterBuilder<T> Or(Func<T, bool> filter)
	{
		var q = this.stack.Pop();


		Func<T, bool> result = (item) =>
		{
			return q(item) || filter(item);
		};


		this.stack.Push(result);
		
		return this;
	}


	public Func<T, bool> Filter => this.stack.First();

	public bool Test(T item)
	{

		return this.stack.First()(item);
	}
}

Hehe, well I guess the first thing you saw, was the stack there, what is the deal with the stack ? We will get to that in a moment, but first let’s see how we can use this wonderful FilterBuilder class.

// case 2
var fb = FilterBuilder<string>.Create(word => word.IndexOf("c") > -1)
					.And(word => word.IndexOf("c") > -1)

// case 3
var fb = FilterBuilder<string>
			.Create(word => word.IndexOf("c") > -1)
				.And(word => word.IndexOf("c") > -1)
			.Or(word => word.IndexOf("b") > -1);

// usage
var result = list.Where(fb.Filter); // or list.Where(wrd => fb.Filter(item) it's a metter of taste 🙂
	

Now, isn’t this cool ? One thing to keep in mind is the composition direction, basically all the operations are processed right to left, and are evaluated after each operation, more or less like math ( order of operations of sorts). I encourage you to put some brake points and have some fun with it.

Now, if you are thinking that this might also go out hand quite quick you could also compose it:

	var list = new List<string>(){
		"a", "ab", "abc", "abcd", "abcde", "bx", "dcba", "dcb", "d",
	};
	
	var fb = FilterBuilder<string>
				.Create(word => word.IndexOf("c") > -1)
					.And(word => word.IndexOf("c") > -1)
				.Or(word => word.IndexOf("b") > -1);
				
	var fb2 = FilterBuilder<string>
				.Create(word => word.StartsWith("a"));

	var fbFinal = FilterBuilder<string>
				.Create(fb.Filter).And(fb2.Filter);

Okay, you admit you like this, but still what is the deal with the stack you ask ?

Well… to be honest it is more or less a small hack, in order to do functional composition, and dynamically compose and evaluate this we had to use some sort of recursion, and “Func” are, well … reference types, and we would have gotten stack traces, so what we did, is use a stack as a buffer, for the new func(func(func()))), and it being a closure would keep all it needs inside its scope, and we just pop the old function from the stack to be garbage collected.

Now, if you are wondering, how exactly is this used dynamically ( build from the database, or from user interactions), here is the example you were looking for, this isn’t production grade, but it might help you get the point:

	// dynamic example
    // this comes from ui / db, serilized however you want
	var containWay = new List<(string, string)>(){
		("and", "c"),
		("and", "d"),
		("or", "b")
	};
	
	FilterBuilder<string> f = null;
	
	foreach(var (rule, letter) in containWay){
		if(f == null){
			f = FilterBuilder<string>.Create(stringRule( letter));
		}else{
			f = rule switch {
				"and" => f.And(stringRule(letter)),
				"or" => f.Or(stringRule(letter)),
				_ => throw new Exception()
			};
		}
	}
	
	var r = list.Where(f.Filter);
	r.Dump(); // did it LinqPad 😀
	
	// helper function for your rules
	Func<string, bool> stringRule(string letter){
		
		return (wrd) => {
			 return wrd.IndexOf(letter) > -1;
		};
	}

So, in this whole post we used just a dummy string list, the beauty of this, is that we use lambdas and we can filter any kind of object, just replace the “string” and ka-boom!

There you have it, a interesting way of building dynamic filters. If you like this kind of content, let me know in the comments.

Processing…
Success! You're on the list.