Click here to Skip to main content
Email Password   helpLost your password?
L LINQDynamicPredicate_demo

Introduction

I'm a big believer in practical examples. So, soon enough, I'll get to one, hold on. Before I do though, I have to admit, when I first saw LINQ-to-SQL I was quite a bit skeptical. I was thinking, "Wow, more syntactic sugar. So it doesn't FEEL like I'm querying anything when I query SQL. BIG DEAL." After all, I'd seen that before.

What I wasn't prepared for was the fact that Microsoft had created a crafty little masterpiece. LINQ basically allows each one of us to extend the language, in a certain manner of speaking. We can take advantage of .NET's JIT-like reflexes (sorry, yes the pun with CAT-like is easy to miss,) by constructing trees of Linq Expressions.

That sounds quite a bit fancier than it is, but the short of it means that I can remove a combinatorial level of manual query writing when I want to combine predicates (tests for rows inclusion in the result set) based on run-time user input.

Background

Using SQL Server is great, as long as you have a great understanding of the T-SQL langauge, and are relatively comfortable with writing it as needed. There are various shortcuts, such as ORM frameworks that make it conceptually easier to work with SQL. That's especially true when writing it out longhand isn't your cup of tea. But when it really comes down to it, anything beyond object persistence in most of these ORM tools is potentially painful, and usually requires some manual SQL code intervention.

SQL is a great back-end for web Apps. In a corporate (politically correctly read 'Enterprise') environment, you like to be able to put out some kind of application that is useful for other corporate employees. One of the things they might like to do would be searching databases. Now, for you, the SQL guru, that's no big deal. When you want to let someone else at it, you really have to make sure you've covered all the bases and tied up all the loose ends (and used too many buzzwords.) I know it's a little vague so far, so, how about that example?!?

Finally, the Example

Let's pretend that you're the IT shop's in house developer. You have the big manager, Sallie, from HR, who'd like to be able to search all of the employees based on any of the available criteria in the corporate database, (which so happens to look a lot like the Northwind database included with SQL Server.)

That doesn't sound so hard, at first... You have an Employees table and you just need to search the fields in there. No problem... So you make a nicely styled, well oiled (and colored) app in the Visual Studio Designer in which you select which of the 18 fields you'd like to search. You've wired up a text box to supply a value, used as a parameter in a SQL select, to data-bind the results grid to either of 18 SqlDataSource's (one for each field.)

Sallie liked the idea, when you presented it, but she says really needs to be able to query based on multiple columns -- in any order and possibly using them all.

Well, that shoots the data-binding solution in the foot. Looks like you'll have to code this up with a command pattern... So you allow the user to pick from a check box list any of the fields that will included in the query, and to specify the criteria. You can do a bunch of <code>switch</code>-ing <code>if</code> logic to figure out what kind of query to render by adding a piece of it at a time, or... (of course you knew I'd get to this,) you can use lambdas and the PredicateBuilder to inject multiple predicates into an expression tree!

Now, let's be clear, I didn't invent the PredicateBuilder, in fact, it's quite a popular little doodad brought to the world by Joseph Albahari. You can see it for free here or check it out in it's natural habitat, the LINQKit, with a few other gadget-like utility classes in the same vein.

Solving this problem without LINQ and the PredicateBuilder basically requires the writing of a query variation for every possible combination of columns, or a way to build a query that works out to the same effect. Compiler writers scoff at the thought of this being considered difficult, but for the rest of us, there's only one way to build and walk a tree that compiles to some other kind of code (SQL in this case) - have someone else do it.

Using the Code

The first notable feature of this example is the user interface. A user will insist that a user interface is all there is in an application. They often refer to a 'feature of the app' as a 'screen,' so getting this part set up reasonably is pretty important.

Since this solution requires that the user be able to select any or all of the columns, we like the idea of a checkbox for inclusion of a filter. The most of the fields in question here are nvarchar's so we can use a String.Contains() lambda for them. The DateTime field, the Birth Date, requires the use of value range comparison, so that filter will have to include a lambda that tests a date for betweeness, (and also two valid date inputs.)

So the individual filter lambdas look like this:

// This kind of lambda, when evaluated, will generate a LIKE clause with wildcards on both sides 
e => e.FirstName.Contains(filterFirstName.Text)

// This kind of lambda, when evaluated, will generate two value comparisons, just like it looks here
e => e.BirthDate.Value >= startDateRange && e.BirthDate.Value <= endDateRange

That's great, you might say, but how do I combine them?

There's two ways to do this, you can do it client side, which means that you're loading ALL the data from SQL and filtering on the web server end. You create an array of lambdas, looping through each one and applying it as a filter to the IQueryable as a Where condition. That can be expensive in server memory. OR you can do it the right way, using PredicateBuilder. Basically LINQ's Where extension to IEnumerable<T> takes a conditional expression as a parameter. This can be a simple lambda, or this can be a complex combination of many lambdas, or other valid LINQ expressions. The PredicateBuilder just makes it easy to combine an undetermined number of them.

From the sample code:

    public IQueryable<Employee> PrepareDataSource() {

            // we start with .True, as in, no filters, get all of them.    
            var predicate = PredicateBuilder.True<Employee>();


            // just inspect the 'checks'... throw in some lambdas

            int emplId = -1;

            // use tryparse to make sure we don't run a bogus query.
            if (cbxUseEmployeeID.Checked &&
                int.TryParse(filterEmployeeId.Text, out emplId) &&
                emplId > 0) {

                // here's how simple it is to add a condition to the query. Still not executing yet, just building a tree.
                predicate = predicate.And(e => e.EmployeeID == emplId);
            }

            if (cbxUseLastName.Checked &&
                !string.IsNullOrEmpty(filterLastName.Text)) {

                // this translates into a LIKE query.
                predicate = predicate.And(e => e.LastName.Contains(filterLastName.Text));
            }

            if (cbxUseFirstName.Checked &&
                !string.IsNullOrEmpty(filterFirstName.Text)) {

                // this translates into a LIKE query.
                predicate = predicate.And(e => e.FirstName.Contains(filterFirstName.Text));
            }

            if (cbxUseTitle.Checked &&
                !string.IsNullOrEmpty(filterTitle.Text)) {

                // this translates into a LIKE query.
                predicate = predicate.And(e => e.Title.Contains(filterTitle.Text));
            }

            DateTime startDateRange = new DateTime(); // default value to avoid 'unassigned use' errors. 
            DateTime endDateRange = new DateTime(); 

            if (cbxUseBirthDate.Checked &&
                DateTime.TryParse(filterBirthDateStart.Text, out startDateRange) &&
                DateTime.TryParse(filterBirthDateEnd.Text, out endDateRange)) {

                // tack on some numeric range testing. I'd have liked to do a between query, but I don't know if
                // there is one that translates in such a way, so we'll just do this:

                predicate = predicate.And(e => e.BirthDate.Value >= startDateRange && e.BirthDate.Value <= endDateRange);
            }

            if (cbxUseAddress.Checked &&
                !string.IsNullOrEmpty(filterAddress.Text)) {

                // this translates into a LIKE query.
                predicate = predicate.And(e => e.Address.Contains(filterAddress.Text));
            }

            if (cbxUseCity.Checked &&
                !string.IsNullOrEmpty(filterCity.Text)) {

                // this translates into a LIKE query.
                predicate = predicate.And(e => e.City.Contains(filterCity.Text));
            }

            if (cbxUseState.Checked &&
                !string.IsNullOrEmpty(filterState.Text)) {

                // this translates into a LIKE query.
                predicate = predicate.And(e => e.Region.Contains(filterState.Text));
            }
            

            if (cbxUsePostalCode.Checked &&
                !string.IsNullOrEmpty(filterPostalCode.Text)) {

                // this translates into a LIKE query.
                predicate = predicate.And(e => e.PostalCode.Contains(filterPostalCode.Text));
            }

            if (cbxUseCountry.Checked &&
                !string.IsNullOrEmpty(filterCountry.Text)) {

                // this translates into a LIKE query.
                predicate = predicate.And(e => e.Country.Contains(filterCountry.Text));
            }

            if (cbxUseHomePhone.Checked &&
                !string.IsNullOrEmpty(filterHomePhone.Text)) {

                // this translates into a LIKE query.
                predicate = predicate.And(e => e.HomePhone.Contains(filterHomePhone.Text));
            }

            if (cbxUseNotes.Checked &&
                !string.IsNullOrEmpty(filterNotes.Text)) {

                // this translates into a LIKE query.
                predicate = predicate.And(e => e.Notes.Contains(filterNotes.Text));
            }

            var results = Config.GetCurrentContext().Employees.Where(predicate);

            // If you are debugging, you can put a breakpoint up there and see the Query by hovering over 'results'.

            // this query is constructed but not yet exectued
            return results;
        }
        

It's so easy with the PredicateBuilder, I think they're trying to make it illegal in several states. (I'd say that is nearly akin to the so-called 'mechanical engineers' getting upset about programmers being called 'software engineers.')

Points of Interest

If you look in the download at the class Employee, you'll notice that I'm taking advantage of the partial nature of the objects that LINQ-to-SQL creates. I added a static method that returns the entire IQueryable<Employee> in the database. Notice I also tagged the partial class with a System.ComponentModel.DataObjectAttribute. I did this so I could populate the columns in the GridView by binding it temporarily to an ObjectDataSource using the GetAll() as the Select method. All the columns appear, I delete the ObjectDataSource, and go on my merry way. Saved myself a little typing.

Something you may not know is that LINQ-to-SQL is VERY intelligent. Yes. Have you ever thought, hmmm this would best be served by a SELECT Foobar where Baz in ('some', 'list', 'of', 'elements') and then thought better of it because obviously LINQ can't generate that? If you have, you'd have been wrong(!), because as it turns out, it can grok it just about the same way that you would. I don't use it in the code download... but see here:

string [] values = new string[] { "Leverling", "Davolio", "Callahan", "Dodsworth" };

// No WAY that'll work! 
predicate = predicate.And(e => values.Contains(e.LastName));

// Yes WAY! It gets translated to  AND Employees.LastName in ('Leverling', 'Davolio', 'Callahan', 'Dodsworth')

I did a little bit of math, and I figured out that if I were to write all possible combinations of the 18 fields in the Employees Table combinatorially, I would have to construct 262,142 unique SQL statements to accommodate each one. That's not to mention an IF statement and a condition block for each! (I know, nobody really does it that way, I hope.)

If you were to write a query builder of your own, you'd have to do a little bit of compiler-like translation. I know, from extensive experience writing software that performs custom tree-based syntactic translation, there are corner cases that you wouldn't expect, and those are what take the longest to find and fix.

The moral of the story is, when you absolutely need flexibility (not necessarily speed) of behavior, runtime translation is the way to go. LINQ-to-SQL is a nice accessible, available, ready made solution. And when someone else has already written something that does that for you, use it. (The author looks at you, while he points at 'LINQ-to-SQL.') Don't roll your own, if you don't have to.

History

You must Sign In to use this message board.
 
 
Per page   
 FirstPrevNext
GeneralDynamic Predicate Builder
Sayed Sajid
22:36 24 Aug '09  
Hi Dave,


Very thanks its a very useful article.

Apart as shown by you, based on the UI selection, there are 262,142 probable combinations on the shown UI.

Am going though the same phase, as we just started implementing linq in our development. Actually I wanted that predefined predicate builder available as said by you.

I would be thankfull in replying back.

Many thanks.

regards
Sajid.
GeneralLinq to Entities
coleydog
16:26 24 Mar '09  
This is great for Linq to Sql however does not build for Linq to Entities.

Using dynamic predicates greatly extends the conditional capabilities of Linq in a simple concise way, can you suggest a work-around when using Linq to entities.

Cheers.
GeneralIs there somethimg like parenthesis
gerecht
0:01 5 Mar '09  
Hi there great article,

in my application i've got to construct a predicate like:

a and b and (c or d)

is this possible?

with kind regards

benjamin
GeneralRe: Is there somethimg like parenthesis
st.dehnert
4:49 10 Mar '09  
you can find the solution on the mentioned side
http://www.albahari.com/nutshell/predicatebuilder.aspx
in the section
nested predicates
GeneralWhere clause not appending to Select statement
sfomate
15:24 19 Feb '09  
Hi Dave,
I've got your sample running fine but when I try to implement my own project doing exactly the same things the where clause is not getting appended to the select statement. I've gone through debug and see that the predicate variable is correctly populated
+          predicate     {f => (True Or Invoke(e => e.City.Contains(value(ASP.regsearch_aspx).txtCity.Text),f))}     System.Linq.Expressions.Expression<System.Func<RegAdmin.RegSearch,bool>>

but the results variable is missing the where clause its value is + results     {SELECT [t0].[Registration Type] AS [Registration_Type], [t0].[Permit_Id], [t0].[Registration Number] AS [Registration_Number], [t0].[Permit Status] AS [Permit_Status], [t0].[Facility Name] AS [Facility_Name], [t0].[Address], [t0].[City], [t0].[Zip Code] AS [Zip_Code], [t0].[Issued], [t0].[Expires]
FROM [dbo].[vw_RegistrationAdminSearchPage] AS [t0]
}     System.Linq.IQueryable<RegAdmin.RegSearch> {System.Data.Linq.DataQuery<RegAdmin.RegSearch>}


Please let me know what could be missing, I've got the config.cs and predicate.cs as well as global.asax copied from your project.

In my DBML I've got just one view for select and one stored proc, that I'm not using.

Thanks in advance.

-Manoj
Generalerror
avidie
8:45 30 Oct '08  
I am getting the error
Error 2 The type arguments for method 'System.Linq.Enumerable.Where(System.Collections.Generic.IEnumerable, System.Func<tsource,bool>' cannot be inferred from the usage. Try specifying the type arguments explicitly.

for following code.
var results = Config.GetCurrentContext().usp_SALEINFO_get().Where(predicate);

usp_SALEINFO_get() is a SQL Stored Procedure.
GeneralRe: error
dave.dolan
9:39 30 Oct '08  
that's not enough to determine your problem. I can't tell you what's wrong with your code from just that line, because it's obviously something to do with how you're constructing the predicate. Perhaps you didn't start with the .True or .False. gotta do that.
GeneralRe: error
avidie
9:44 30 Oct '08  
Thank you for the reply.. here is the complete code..
repeaterSearchInfo.DataSource = SearchParcels (strTown);
repeaterSearchInfo.DataBind();


public IQueryable SearchParcels(string strTown)
{
var predicate = PredicateBuilder.True();
if (!string.IsNullOrEmpty(strTown))
{

predicate = predicate.And(u => u.TOWN == strTown );
}

return mobj.usp_SALEINFO_get().Where(predicate);
}

Predicate builder
using System;
using System.Linq;
using System.Linq.Expressions;
using System.Collections.Generic;

public static class PredicateBuilder
{
public static Expression<t,>> True () { return f => true; }
public static Expression<t,>> False () { return f => false; }

public static Expression<t,>> Or (this Expression<t,>> expr1,
Expression<t,>> expr2)
{
var invokedExpr = Expression.Invoke (expr2, expr1.Parameters.Cast ());
return Expression.Lambda<t,>>
(Expression.Or (expr1.Body, invokedExpr), expr1.Parameters);
}

public static Expression<t,>> And (this Expression<t,>> expr1,
Expression<t,>> expr2)
{
var invokedExpr = Expression.Invoke (expr2, expr1.Parameters.Cast ());
return Expression.Lambda<t,>>
(Expression.And (expr1.Body, invokedExpr), expr1.Parameters);
}
}

GeneralRe: error
dave.dolan
10:06 30 Oct '08  
you aren't specifying the type you're querying against. The True() must actually be True I can't tell what kind of object you have in your code at all. You may have to find out what type the stored procedure is compiled to output. Then just declare your type as IQueryable. I can't remember off the top of my head how it generates that for stored procedures, but it's some long winded type name Smile Just look at the generated code from your dbml, and find tha t function, and see what type it returns. Then just do it against that type, instead of just leaving the type out altogether.
GeneralRe: error
avidie
10:14 30 Oct '08  
Seems the message is truncating the true tag "<>"

I have the DBML file as

[Function(Name="dbo.usp_SALEINFO_get")]
public ISingleResult usp_SALEINFO_get()
{
IExecuteResult result = this.ExecuteMethodCall(this, ((MethodInfo)(MethodInfo.GetCurrentMethod())));
return ((ISingleResult)(result.ReturnValue));
}
public partial class usp_SALEINFO_getResult
{...
........
}

I am specifying the type as True as usp_SALEINFO_getResult >();
"var predicate = PredicateBuilder.True< usp_SALEINFO_getResult>();"

the stored procedure name is usp_SALEINFO_get
return mobj.usp_SALEINFO_get().Where(predicate);

GeneralRe: error
dave.dolan
10:17 30 Oct '08  
you'll have to do PredicateBuilder.True <ISingleResult <usp_SALEINFO_getresult > > The whole thing. It's like a data table. I multiple result is actually like a dataset (multiple tables.) The name of the interface is confusing.
GeneralRe: error
avidie
11:12 30 Oct '08  
No luck Dave. I tried all possible combinations like

var predicate = PredicateBuilder.True <(ISingleResult usp_SALEINFO_get()) > ();
var predicate = PredicateBuilder.True <(ISingleResult) > ();

it gives syntax errors as its not recognising ISingleResult word

GeneralRe: error
dave.dolan
11:30 30 Oct '08  
You have to "using System.Data.Linq", then you can use ISingleResult :P You will not get away without using it. period. you must use ISingleResult. The class will be something like: ISignleResult<NameOfTheStoredProcResult>. You don't just do ISingleResult<NameOfSToredPRoc>
QuestionRe: error
avidie
11:41 30 Oct '08  
Thank you. It worked. but another issue. The predicate doesn't have the reference for the result values, in this case it can't find u.Town

predicate = predicate.And( u => u.TOWN.Contains ( strTown) );
AnswerRe: error
dave.dolan
11:47 30 Oct '08  
That just means that somewhere you forgot to specify the ISingleResult<whatever>
GeneralRe: error
avidie
13:11 30 Oct '08  
Tried all the things, doesn't work. can you give a try with a simple stored proc on Linq and Predicate construction and post the code. Can't find one any where. Thanks in advance.
GeneralRe: error
dave.dolan
17:01 30 Oct '08  
haha, well it's obvious that you haven't tried all things. if you had, one of them would have worked. I'll see if I can find some time to do this, but I can't promise you anything at the moment.
GeneralVB.Net Version
Jing Celeste
18:15 15 Oct '08  
Hello Sir!

Im a newbie in .Net... I would appreciate if you could give a VB Code..

Thanks a lot.
GeneralLikable. Very, very likable.
Pete O'Hanlon
12:28 27 Aug '08  
Have yourself a spanky danky 5 sir.

Deja View - the feeling that you've seen this post before.

My blog | My articles



QuestionSorting
Doncp
13:41 20 Aug '08  
Very powerful!

How do you use OrderBy to sort
the result?

Thanks,
Don
AnswerRe: Sorting
dave.dolan
9:40 30 Oct '08  
tack an orderby on the end of it. there's orderby and then there's thenby. This is just a LINQ thing, not anything particular to do with the predicate builder.
GeneralThoughts
MR_SAM_PIPER
17:18 18 Aug '08  
Technically for the example you've provided you don't need PredicateBuilder at all.

Statements like this:

predicate = predicate.And(e => e.Region.Contains(filterState.Text));
var results = Config.GetCurrentContext().Employees.Where(predicate);

Can be expressed like this:

var query = from e in Config.GetCurrentContext().Employees select e;
if (doRegionFiltering)
{
query = query.Where(e => e.Region.Contains(filterState.Text));
}

if (doNameFiltering)
{
query = query.Where(e => e.LastName.Contains(filterLastName.Text));
}

etc...

As long as you are ANDing your search conditions together, this syntax works in exactly the same manner, as you are just building an expression tree for each call to Where(), that will be evaluated when the query is enumerated.

If you need to OR conditions together outside of one Where() then PredicateBuilder becomes useful, as it easily lets you OR existing predicates together (something that is more complex to do using the LINQ API directly).
GeneralRe: Thoughts
dave.dolan
17:33 18 Aug '08  
Yes you're correct, I was thinking in the abstract as in, for any possible conditions, but I only picked examples using the AND. I didn't even notice that until just now when you've pointed it out. I originally had set out to make a radio next to each selecting 'or' vs 'and' but I guess I neglected to put that bit in before publish time. Anyway, you're right.
GeneralRe: Thoughts [modified]
Jeremy Thomas
18:13 18 Aug '08  
I was just about to post to say the same thing:

public IQueryable<Employee> PrepareDataSource()
{
IQueryable<Employee> results = Config.GetCurrentContext().Employees;

if (cbxUseHomePhone.Checked && !string.IsNullOrEmpty(filterHomePhone.Text))
{
// this translates into a LIKE query.
results = results.Where(e => SqlMethods.Like(e.HomePhone, filterHomePhone.Text));
//results = results.Where(e => e.HomePhone.Contains(filterHomePhone.Text));
}
return results;
}

Another thing you could do change the contains to a SqlMethods.Like for more complicated like clauses.

But thanks for the article though, over the last week I have been trying to find examples of how to dynamically combine (i.e, include/exclude) static where predicates.
Most blogs on Dynamic LINQ talk about dynamically creating the where predicate itself which is a level beyond what I am after.
Of course I came across PredicateBuilder but their website only has snippets but your example shows exactly the scenario I want to achieve - searching on user selected criteria.

btw
Other examples ive since come across:
www.jstawski.com/archive/2008/05/21/dynamic-construction-of-a-linq-expression.aspx[^]
Chapter 5 of linqinaction.net[^]
http://blogs.msdn.com/vbteam/archive/2007/08/29/implementing-dynamic-searching-using-linq.aspx[^]
http://rocksthoughts.com/blog/archive/2008/04/10/linq-to-sql-dynamic-queries-3-ands--ors-together.aspx[^]

Jeremy Thomas

modified on Tuesday, August 19, 2008 2:43 PM

GeneralYour timing ...
Jammer
23:46 13 Aug '08  
... is impecable ...

I've just been implementing MVC in an app and the next step was to look into building rather complex Predicates in order to power the filtering on an ICollectionView ...

I'll be reading this very intently!

Thanks,

Jammer
if(glass == Glass.Empty){GoToBarAndOrderMoreBeer();}
else {Drink();}
Smile
My Blog | Article(s)


Last Updated 14 Aug 2008 | Advertise | Privacy | Terms of Use | Copyright © CodeProject, 1999-2010