Click here to Skip to main content
13,668,332 members
Click here to Skip to main content
Add your own
alternative version

Stats

5.4K views
202 downloads
19 bookmarked
Posted 20 Apr 2018
Licenced CPOL

LINQ Part 3: An Introduction to IQueryable

, 20 Apr 2018
Rate this:
Please Sign up or sign in to vote.
Part 3 in the LINQ series, this provides an introduction to IQueryable, IQueryProvider, and LINQ expression trees.

Introduction

In this series, we've explored IEnumerable and taken a look at the standard methods that extend this interface. Together these form a small, but crucial part of LINQ, known as LINQ to Objects.

Another important aspect of LINQ is its ability to query other data sources (e.g. databases) where the information does not reside in local memory. In fact, it may even exist on an entirely different machine.

To support this very different requirement, LINQ introduces the concepts of query provider (System.Linq.IQueryProvider), expression tree (System.Linq.Expressions.Expression), and queryable sequence (System.Linq.IQueryable). This article will explore these concepts.

Background

With LINQ to Objects, LINQ can simply provide standard methods that operate directly on the sequences that are queried. With other data sources, this would be inefficient.

For example, for a database, we want to make as few round-trips to the server as possible. Also, we want to request as little information as possible. Finally, we want to take advantage of all of the "smarts" of the database product.

To facilitate this, LINQ introduces the following new concepts:

  • Query provider (IQueryProvider) - This is specialized software that can interpret a query, so that it efficiently utilizes the underlying resources.
  • Expression tree (Expression) - A tree is formed from elements of a query. This tree is later analyzed by the query provider. This should be familiar to anyone who has ever written a parser or compiler.
  • Queryable sequence (IQueryable) - This is the approximate equivalent to IEnumerable in LINQ to Objects. It simply pairs a query provider with an expression tree.

As with IEnumerable, LINQ provides a set of standard methods, defined in the System.Linq.Queryable class. These methods all extend IQueryable. They all share identical names and near identical syntax with their counterparts in System.Linq.Enumerable.

While this reduces the learning curve for developers, it is also deceptive. While conceptually similar, these methods accomplish their goal in an entirely different manner: they build the expression tree.

This is the third in a series of articles on LINQ. Links to other articles in this series are as follows:

LINQ Expression Trees

Most of the methods in System.Linq.Enumerable simply create a new instance of IEnumerable that wraps the one on which it operates. Since the underlying sequence generally exists in memory, or is easily acquired, there is no real concern about the mechanism by which it is fetched.

The methods in System.Linq.Queryable operate in a different manner. Each consumes an IQueryable, which is simply a pairing of an expression tree (Expression) with a query provider (IQueryProvider). Each method produces a new IQueryable, where the expression tree has been altered to include additional elements of the query.

Let's consider a simple query:

Source.Students.Where(student => student.LastName == "Marx")

The first IQueryable we need to consider is Source.Students. In this case, the expression tree (IQueryable.Expression) consists entirely of a single node (ConstantExpression). The value of this node (ConstantExpression.Value) is simply the queryable itself.

The next IQueryable we need to consider is the one returned by the Where method. Here things get a lot more complex. In this case, the expression tree (IQueryable.Expression) consists of many nodes. Visually, it appears as follows:

The nodes in the tree are as follows:

  1. The call to the Where method, which takes two arguments, each of which appears in the tree.
  2. The original IQueryable upon which the Where method operates.
  3. A container for the Lambda expression that provides the predicate for the Where method. Note: It seems odd (even to me) that this node is required.
  4. The Lambda expression (student => student.LastName == "Marx") that provides the predicate for the Where method.
  5. The parameter for the Lambda expression (student).
  6. The "equals" comparison that is evaluated within the Lambda expression (student.LastName == "Marx").
  7. The left hand operand for the "equals" comparison, which accesses a property/member (LastName).
  8. The right hand operand for the "equals" comparison, the constant "Marx".
  9. The parameter for the property/member access (student). This is the specific instance where the property value is found.

What Good is an Expression Tree?

So, now we have a nifty expression tree. What use is it?

This is where the query provider (IQueryProvider) comes into the picture. It is responsible for "executing" the query described by the expression tree.

For example, in the case of a database provider, it may do quite a bit. First, it needs to translate the expression tree into a SQL query. So, in our previous example, this may be something like:

SELECT * FROM Students WHERE LastName = 'Marx'

It might then need to execute the query on the database server, fetch the results, instantiate objects for each of the rows, and provide an enumerable of these objects.

A Quick Review

From this article, we've learned the following:

  • The extension methods in System.Linq.Queryable operate on instances of IQueryable, by building an expression tree (Expression).
  • IQueryable simply pairs the expression tree (Expression) with a query provider (IQueryProvider).
  • The query provider (IQueryProvider) is responsible for interpreting the expression tree, executing the query, and fetching the results.

The key word here is "interpreting". Because the query provider (and its underlying resource) is likely more limited than the C# language, there are limitations to what can appear within the query.

In general, the Lambda expressions are limited to fairly vanilla operations: comparisons, string manipulations, and projection (creation of new instances from existing property values). The exact limitations are dependent on the query provider itself.

If you're merely consuming a LINQ queryable, what you've learned so far, is about all you need to understand.

The remainder of this article dives a bit deeper. Feel free to learn more, or skip it, depending on your specific needs.

Exploring More Complex Queries

So far, we've only looked at a very simple query. As the query grows in complexity, the expression tree becomes rather large. To facilitate exploring more complex queries, an application (QueryableFun) is included with this article.

Mostly, I recommend simply playing with the application. It allows the user to run some pre-defined queries. These are selected from the "Queries" drop down list, which appears in the upper right hand corner of the window, in the tool strip.

To execute a query, simply click the "Run" button ().

The application consists of two panels: the expression tree (left) and a tab control (right). Within the tab control, you'll find four tabs. There are as follows:

  • Expression - Enter or view C# LINQ expressions here.
  • Properties - When you click on a node in the expression tree, this will display the properties for that node.
  • Results - A data grid with the results of the query.
  • Errors - Any compilation errors that occur while evaluating the expression.

A screen capture of the application appears below. Please understand that this is simply a learning application. It is not intended for any production use. As such its quality is lower than what I would normally produce.

For serious explorations of LINQ, I highly recommend LINQPad. I have zero association with the company. I simply relied on this, quite heavily, when I was teaching myself LINQ. The application comes in both free and paid versions.

IQueryable Members

While this is the most important interface, it is also the simplest. Its primary purpose is to pair an expression tree with a query provider. In fact, it is so simple that a single implementation could satisfy most query providers.

It has only four members, which are as follows:

Member Name Description
Expression This is the expression tree for this queryable.
ElementType The type of element that constitutes the sequence.
Provider The query provider responsible for interpreting and executing the query.
GetEnumerator<TElement> Gets a strongly-typed enumerator (IEnumerator<TElement>) for this queryable. This usually calls the query provider to interpret and execute the query.
GetEnumerator Gets a weakly-typed enumerator (IEnumerator) for this queryable. Often, this simply invokes GetEnumerator<TElement>().

IQueryProvider Members

While the implementation of this interface is generally quite complex, the interface itself is quite simple. It allows a consumer to create queryables and execute expressions. It has only four members:

Member Name Description
CreateQuery() Creates a weakly-typed queryable (IQueryable) for the specified expression.
CreateQuery<TElement>() Creates a strongly-typed queryable (IQueryable<TElement>) for the specified expression.
Execute() Executes the specified expression and returns a weakly-typed result.
Execute<TResult>() Executes the specified expression and returns a strongly-typed result.

QueryableFun Modules

The QueryableFun application is mostly intended to provide a flavor of what expression trees look like. While it is not a very serious effort, it does demonstrate a few clever tricks.

  • Roslyn - It provides an example of compiling and executing C# script using Microsoft's Roslyn technology.
  • IQueryable - It provides an example of a very simple IQueryable implementation. Instead of the complexity of evaluating and executing a query against an external resource, this implementation simply wraps an underlying collection.
  • IQueryProvider - It provides an example of a very simple IQueryProvider implementation. This implementation simply alters the expression tree to replace the original queryable with a queryable for an underlying collection. It then tricks LambdaExpression.Compile() into doing all the heavy lifting.
  • Toy data - It provides some "toy" data for LINQ queries, so that no database or more complex resource is required.

The modules in this application are as follows:

Module Name Description
EnumerableExtensions Extensions to IEnumerable that simplify finding getting the element type for the enumerable.
ExpressionTreeBuilder An ExpressionVisitor that populates a Windows Form TreeView.
ExpressionTreeRemediator An ExpressionVisitor that swaps out nodes containing the original IQueryable for a new IQueryable for the underlying collection.
InterestingQueries A set of pre-defined queries that demonstrate some common use cases.
MainAboutBox A fairly standard "Help About..." dialog.
MainForm The User Interface for the application.
Program The main launching point for the application.
QueryableEnumerable An implementation of IQueryable that simply wraps an underlying collection.
QueryableEnumerableProvider An implementation of IQueryProvider that uses LambdaExpression.Compile() to do all the heavy lifting.
Source A static source for LINQ queries, which can be easily referenced in user-provided expressions

In the Transeric.Queryable project, which is also a part of this application, "toy" data is provided in the form of simple POCO classes and collections based upon them.

Further Reading

For further reading, see the following:

IQueryable<T> Interface
https://docs.microsoft.com/en-us/dotnet/api/system.linq.iqueryable-1

IQueryProvider Interface
https://docs.microsoft.com/en-us/dotnet/api/system.linq.iqueryprovider

Expression Class
https://docs.microsoft.com/en-us/dotnet/api/system.linq.expressions.expression

Queryable Class
https://docs.microsoft.com/en-us/dotnet/api/system.linq.queryable

LINQ to Objects
https://docs.microsoft.com/en-us/dotnet/csharp/programming-guide/concepts/linq/linq-to-objects

LINQ to SQL
https://docs.microsoft.com/en-us/dotnet/framework/data/adonet/sql/linq/index

LINQPad
https://www.linqpad.net/

History

  • 4/20/2018 - The original version was uploaded
  • 4/21/2018 - Added links to other articles in this series
  • 4/23/2018 - Updated exe and src to handle singleton LINQ methods
  • 4/24/2018 - A couple of minor cosmetic and grammar corrections
  • 4/25/2018 - Added link to fourth article in series

License

This article, along with any associated source code and files, is licensed under The Code Project Open License (CPOL)

Share

About the Author

Eric Lynch
Software Developer (Senior)
United States United States
No Biography provided

You may also be interested in...

Comments and Discussions

 
QuestionMissing Pin
Alen Toma23-Apr-18 9:54
memberAlen Toma23-Apr-18 9:54 
AnswerRe: Missing Pin
Eric Lynch23-Apr-18 11:57
memberEric Lynch23-Apr-18 11:57 
GeneralRe: Missing Pin
Alen Toma23-Apr-18 12:13
memberAlen Toma23-Apr-18 12:13 
QuestionMy vote of 5 Pin
Member 1020002223-Apr-18 7:53
memberMember 1020002223-Apr-18 7:53 
AnswerRe: My vote of 5 Pin
Eric Lynch23-Apr-18 8:18
memberEric Lynch23-Apr-18 8:18 
QuestionNote: It seems odd (even to me) that this node is required. Pin
Marc Clifton23-Apr-18 1:15
protectorMarc Clifton23-Apr-18 1:15 
AnswerRe: Note: It seems odd (even to me) that this node is required. Pin
Eric Lynch23-Apr-18 5:34
memberEric Lynch23-Apr-18 5:34 

General General    News News    Suggestion Suggestion    Question Question    Bug Bug    Answer Answer    Joke Joke    Praise Praise    Rant Rant    Admin Admin   

Use Ctrl+Left/Right to switch messages, Ctrl+Up/Down to switch threads, Ctrl+Shift+Left/Right to switch pages.

Permalink | Advertise | Privacy | Cookies | Terms of Use | Mobile
Web01-2016 | 2.8.180820.1 | Last Updated 20 Apr 2018
Article Copyright 2018 by Eric Lynch
Everything else Copyright © CodeProject, 1999-2018
Layout: fixed | fluid