Click here to Skip to main content
14,550,580 members

How to Implement Full-Text Search in .NET Application with Elasticsearch

Rate this:
5.00 (7 votes)
Please Sign up or sign in to vote.
5.00 (7 votes)
6 May 2020CC (BY-ND 3.0)
Learn how to read and write documents with custom full-text queries in C# using NEST
In this simple tutorial, I will provide a simple demo to read and write documents to Elasticsearch and add a full-text search feature to C# applications. Elasticsearch is a distributed, open-source search engine that manages all kinds of data. Learn how simple it is to integrate into your ASP.NET application!

Why Elastic Search?

Elasticsearch is a distributed, open-source search engine that manages all kinds of data. But why is Elasticsearch the best solution for full-text?

I started playing with full-text search on my thesis in 2009 when I had to implement a searching algorithm for a data recommendation system. The experience was very educational, but starting from scratch was a bloodbath. The first opportunity I got to use a library, I tried out Apache Lucene to implement full-text search. As application architecture becomes complex (i.e., multiple servers that need to share indexed data), Solr provided a scalable solution. It is an opensource API-based search engine (based on Lucene). After using these libraries, I used Elasticsearch which is now the market leader. It comes with an on-premise free version and is available on the cloud. It scales from simple installations to huge environments. So the question is: why not?

The Sample Application

I used with success elastic search on production and on my opensource headless cms, RawCMS. To show how Elasticsearch works, I created a sample application where there are the two main features implemented:

  • Creating an index and adding data into it
  • Reading data using full-text queries

The example code is available on GitHub. To run and test it, just download, compile, and execute:

dotnet ElasticSearchTest.dll create -f divina_commedia.txt -h http://localhost:9300
> Index created in 30338ms with 14006 element.

dotnet ElasticSearchTest.dll search -i divinacommediatxt 
                                    -h http://localhost:9300 -q "dante AND virgi*"
> Searching for $dante AND virgi*
> "Dante, perché Virgilio se ne vada,
> "Dante, perché Virgilio se ne vada,

The console application uses the library ConsoleLineParser for parsing input in a human way. So I just added two classes for verb attributes and a mapping between the console verbs and the code to run.

private static void Main(string[] args)
{
    object x = CommandLine.Parser.Default.ParseArguments<CreateOptions, SearchOptions>(args)
      .MapResult(
        (CreateOptions opts) => DoCreate(opts),
        (SearchOptions opts) => DoSearch(opts),
        errs => 1);
}

Basing on the verb the user entered (search or create), the corresponding procedure is launched passing the arguments.

How to Write Documents

The writing part is quite easy. In fact, for Elasticsearch, there are two options. Using the low-level framework, you can just get a wrapped implementation of the elastic API. This helps to avoid manual binding and composing JSON payloads. However, if you want to play with data, a good option is NEST. NEST is the high-level framework and, if you are an expert with ORMs, this shouldn’t be any surprise.

You just have to create classes for documents you want to save, and define how to save properties using annotations and invoke the save API.

I don’t think this is too complex, and it sounds more like an ordinary routine. Here is the code snippet for the class definition. In this example, I used just one field per document that contains a line of text.

public class LogDocument
{
    public Guid Id { get; set; } = Guid.NewGuid();
    public string Body { get; set; }
}

The next step is to create the index. In this step, we relate the index with the class. This can be done manually, specifying storing settings, or using automapping. In our example, the “Id” field is automatically mapped to the unique identifier of the document.

client.Indices.Create(indexName, c => c
                .Map<LogDocument>(m => m
                    .AutoMap<LogDocument>()
                )
            );

Finally, we have the most stupid part of the code: writing data. Because we want to save all the poem’s lines into many documents, one per row, we have just an iteration.

string[] lines = File.ReadAllLines(filepath);

int items = 0;
Parallel.ForEach(lines, (line) =>
{
    if (!string.IsNullOrWhiteSpace(line))
    {
        client.CreateDocument<LogDocument>(new LogDocument()
        {
            Body = line.Trim()
        });
        items++;
    }
});

Please note that using the API over an external system, we can use a parallel construct to improve performance.

How to Query Documents

This part is quite simple and very clear — at least I hope. The basic way to access documents is by using regular fluent LINQ syntax. This is the basic usage, and I prefer to focus this demo on the not-so-well- documented use case of searching full-text data.

Elastic allows using a raw query to find over field data. To do this, you can use the right overload of Search method, configuring the raw query:

var searchResponse = client.Search<LogDocument>(s => s
                        .Size(10)
                        .Query(q => q.QueryString(

                            qs => qs.Query(searchStr)
                            .AllowLeadingWildcard(true)
                            )
                        )
                    );

var docs = searchResponse.Documents;

What to Take Home

Elasticsearch is the leading search engine solution. It provides applications rich features like full-text search or document indexing. It can be used as a service or on-premise. In either case, it is quite simple to configure for basic usage.

The NEST framework allows us to store and access Elasticsearch like it was a simple database through LINQ, and this makes everything very simple.

For a complex scenario where you want to let end users write their queries, you can use a raw query and map results to classes.

Resources

History

  • 6th May, 2020: Initial version

License

This article, along with any associated source code and files, is licensed under The Creative Commons Attribution-NoDerivatives 3.0 Unported

Share

About the Author

Daniele Fontani
Chief Technology Officer
Italy Italy
I'm senior developer and architect specialized on portals, intranets, and others business applications. Particularly interested in Agile developing and open source projects, I worked on some of this as project manager and developer.

My programming experience include:

Frameworks \Technlogies: .NET Framework (C# & VB), ASP.NET, Java, php
Client languages:XML, HTML, CSS, JavaScript, angular.js, jQuery
Platforms:Sharepoint,Liferay, Drupal
Databases: MSSQL, ORACLE, MYSQL, Postgres

Comments and Discussions

 
SuggestionBeware the cost Pin
compiler11-May-20 7:11
Membercompiler11-May-20 7:11 
PraiseInteressant Pin
Júnior Pacheco6-May-20 10:02
professionalJúnior Pacheco6-May-20 10:02 

General General    News News    Suggestion Suggestion    Question Question    Bug Bug    Answer Answer    Joke Joke    Praise Praise    Rant Rant    Admin Admin   

Use Ctrl+Left/Right to switch messages, Ctrl+Up/Down to switch threads, Ctrl+Shift+Left/Right to switch pages.

Article
Posted 6 May 2020

Stats

4.1K views
14 bookmarked