A Search Engine in Your Pocket – Introducing dtSearch on Android

.NET developers, please see also Faceted Search with dtSearch by this author.

Part 1: Faceted Search with dtSearch (using SQL and .NET)
Part 2: Turbo Charge your Search Experience with dtSearch and Telerik UI for ASP.NET

• Download SimpleNotepad.zip - 184 KB

Introduction

I’m one of these people who likes to experiment with using different phones, and learning how to program on each of them. I was working on a simple project on Android recently and determined that my app needed to be able to support a full-text search. I wanted to be able to search across my content and deliver a cool result list like I would expect on a website. I know ... it’s the web developer in me coming out as I work on a phone project. Nevertheless, as I started examining the problem, I found that my friends at dtSearch have a brand new version of their search library available for Android. I jumped on the opportunity to use an already familiar tool in a new setting.

Previously, I had used dtSearch to index information about products on a web page and deliver highlighted results. In this case, I’m going to show you a demo using a simple Notepad example app that will allow me to search across the notes that I have stored and return a list of results on my Android device.

Getting Started – Referencing the Library

In order to use the dtSearch library with Android, your app must meet these requirements:

• Android API 9 or greater (Gingerbread)
• Intel or ARM processor

Next, you need to place the dtSearch.jar and native library files in your libs/ folder of your project. In my case, since I was using Gradle and Android Studio, I placed the jar file in the libs folder and the native libraries in my src/main/jniLibs folder. The library will work with Eclipse, and you simply place the jar and native libraries in the libs/ folder of the project.

Folder locations for Eclipse	Folder locations for Android Studio

With the files placed, I added the following entry into my dependencies section of app/build.gradle:

dependencies {
   compile files('libs/dtSearchEngine.jar')
}

This tells the java compiler to include the search engine reference when building the application.

Additional Assets – Font Awesome

For this sample project, I also included Font Awesome so that I could generate some decent looking buttons without much work. I added the fontawesome-webfont.ttf file to my assets folder, the font_awesome.xml resources to my res folder and two java files with the definition of the FontAwesome views to my package path.

Now, I can create great looking buttons with XML syntax like the following:

<com.FontAwesome.Example.ButtonAwesome

            android:layout_width="wrap_content"

            android:layout_height="wrap_content"

            android:text="@string/fa_plus"

            android:id="@+id/addButton"

            android:clickable="true" />

        <com.FontAwesome.Example.ButtonAwesome

            android:layout_width="wrap_content"

            android:layout_height="wrap_content"

            android:text="@string/fa_search"

            android:id="@+id/findButton"/>

I get to use the familiar fa_* syntax to reference the icons, just as I could in CSS. My buttons look like the following:

My Note Object

To make this note object as simple as possible for this demo, I have included an Id, Title, Content, Created, and Modified fields. The content of my Note.java file looks like this:

import android.database.Cursor;
import java.util.Date;

public class Note {

    private int id;
    private String title;
    private String content;
    private Date created;
    private Date modified;

    public Note() {}

    public Note(int id, String title, String content, Date created, Date modified) {
/** Omitted for brevity **/
    }

    public Note(Cursor c) {

/** Omitted for brevity **/

    }

    public int get_ID() {
        return id;
    }

    public void set_ID(int id) {
        this.id = id;
    }

    public String get_Title() {
        return title;
    }

    public void set_Title(String title) {
        this.title = title;
    }

    public String get_Content() {
        return content;
    }

    public void set_Content(String content) {
        this.content = content;
    }

    public Date get_Created() {
        return created;
    }

    public void set_Created(Date created) {
        this.created = created;
    }

    public Date get_Modified() {
        return modified;
    }

    public void set_Modified(Date modified) {
        this.modified = modified;
    }

    @Override
    public String toString() {
        return this.title;
    }

}

Configuring the Search Index

For this sample, I want to build and index notes from my notepad when a save operation of a note occurs. As is the case with the .NET and Java versions of the dtSearch engine, you need to start configuring search with a DataSource. The DataSource is a class that handles the acquisition and formatting of content for the dtSearch engine to index.

I created a NoteDataSource that implements the com.dtsearch.engine.DataSource interface. Inside of this class, I created a reference to my NoteRepository object that manages the simple interactions with the SQLite database that stores my notes:

public class NoteDataSource implements DataSource {

    private final Cursor _NotesCursor;
    private NoteRepository _NoteRepository;
    private int _RecordNumber = 0;
    private Note _CurrentNote;
    private int _TotalRecords;

    public NoteDataSource(Context ctx) {

        _NoteRepository = new NoteRepository(ctx);

        try {
            _NoteRepository.open();
        } catch (Exception ex) {
            ex.printStackTrace();
        }

        _NotesCursor = _NoteRepository.SelectAllNotes();

    }

}

The DataSource has methods for the search indexer to be able to navigate and consume the data you want it to index.

The first method to configure is called rewind and it is called when the search indexer is preparing to begin indexing my data. It expects a Boolean value to be returned to indicate success. In my case, I reset my record number to zero, move the database cursor to the first record and count the total number of records in my Note table:

@Override
public boolean rewind() {

    _RecordNumber = 0;
    _NotesCursor.moveToFirst();
    _TotalRecords = _NoteRepository.SelectAllNotes().getCount();

    return true;
}

The next method to set up is getNextDoc() – this method is how the search indexer will iterate over the collection of data to present. This method will be called each time that the indexer is ready to work with a new record. My method implementation is very simple:

@Override
public boolean getNextDoc() {

    if (_RecordNumber == _TotalRecords) return false;

    Log.w("GetNextDoc", "Totalrecords: " + String.valueOf(_TotalRecords) + " RecordNumber: " + String.valueOf(_RecordNumber));

    // Get the current note
    _RecordNumber++;
    _CurrentNote = new Note(_NotesCursor);
    _NotesCursor.move(1);

    return true;

}

In this method, the indexer is inspecting the returned Boolean value to determine if it has more records to iterate over the data source. I return false to indicate that there are no more records when all records have been processed. The _CurrentNote is an object that I load from my SQLiteCursor using a custom constructor method that knows how to parse the SQLite backing table.

These were easy methods to configure as they’re simply managing connections to my backing database. The next methods to configure have to do with the fields in the Note objects that I am indexing.

@Override
public String getDocText() {

    // The text of the note to index
    return _CurrentNote.get_Content();

}

@Override
public String getDocFields() {

    // Fields and content of the note to index for faceted search
    return "";
}

@Override
public String getDocName() {
    return String.valueOf(_CurrentNote.get_ID());
}

@Override
public String getDocDisplayName() {
    return _CurrentNote.get_Title();
}

@Override
public Calendar getDocModifiedDate() {

    Calendar calendar = Calendar.getInstance();
    calendar.setTime(_CurrentNote.get_Modified());
    return calendar;

}

@Override
public Calendar getDocCreatedDate() {
    Calendar calendar = Calendar.getInstance();
    calendar.setTime(_CurrentNote.get_Created());
    return calendar;
}

These field getters should be fairly straightforward, just getting data and formatting it for the indexer. The DocFields is a property that will allow me to (optionally) make available faceted searching. If I were preparing the index for this, I would have output the title and content fields with the required tab delimiters between the fields and their values. The DocName field is an opportunity to store a unique piece of information about the Note so that I can reference it later. In this case, it makes perfect sense to index the primary key of my notes table, the ID. Finally, the DocDisplayName is a “human-friendly” term that can be stored and presented as part of a collection of results from a search.

Building the Search Index

With the DataSource configured, I can write a simple method to rebuild the search index. I could get fancy and write methods to handle incremental updates when notes are updated, added, and deleted. However, this notepad is so small that it’s trivial to rebuild the index each time. Therefore, I will call this Rebuild method any time a save operation takes place in my app.

I added a public static method to my NoteDataSource class to centralize my logic for building the index. The first thing to know about the index, and the first tricky problem I ran into as a web developer entering the Android world, was that I needed to allocate a folder on the device to store my search index. I identified and created my folder in a static getter method called getIndexLocation:

public static File getIndexLocation(Context ctx) {

    File thisDir = null;
    try {
        thisDir = ctx.getDir("index", Context.MODE_PRIVATE);
         } catch (Exception ex) {
        ex.printStackTrace();
    }
    return thisDir;

}

Simple! This code just configures a private folder called index under the current app’s data folder. I then configure an IndexJob to consume my NoteDataSource and write the index into my new index folder.

public static void Rebuild(Context ctx) {

        IndexJob job = new IndexJob();
        NoteDataSource source = new NoteDataSource(ctx);
        job.setDataSourceToIndex(source);
        job.setIndexPath(getIndexLocation(ctx).getAbsolutePath());
        job.setActionCreate(true);
        job.setActionAdd(true);
        job.setCreateRelativePaths(true);
        job.setIndexingFlags(IndexingFlags.dtsIndexCacheTextWithoutFields | IndexingFlags.dtsIndexCacheText);

        job.execute();

        if (job.getErrors() != null && job.getErrors().getCount() > 0) {
            JobErrorInfo errors = job.getErrors();
            Log.w("Index Build", errors.getMessage(0));
        }

    }

To note in this method, I configured the IndexJob to create the index and add data to it. The notes are indexed without using the DocFields method, and the text returned on DocText are indexed. Finally, the job is executed. I added a bit at the end to trap and report any errors during the process.

Searching the Index

With the index built, I crafted a simple search activity with a TextView, another FontAwesome button, and a ListView to contain my results.

I wired up the onclick of the button to trigger a method called DoSearch on the NoteDataSource that would search the index and bind the results to the ListView. This method looks like the following:

public static List<Note> DoSearch() {

    SearchJob job = new SearchJob();
    job.setIndexesToSearch(NoteDataSource.getIndexLocation(this).getAbsolutePath());
    job.setMaxFilesToRetrieve(10);
    job.setRequest(get_SearchText());
    job.setTimeoutSeconds(3);
    job.execute();

    return TransformResultsToNotes(job.getResults());

}

A SearchJob is created and configured to search the folder defined in my NoteDataSource as the index location. At the most, 10 records will be retrieved, and the text to search for is set with the setRequest method. I configured the search to timeout after 3 seconds, which is an eternity for a mobile app, and then executed the search.

The results of the search are transformed into Note objects with the TransformResultsToNotes method:

private static List<Note> TransformResultsToNotes(SearchResults results) {

    List<Note> foundNotes = new ArrayList<Note>(results.getCount());
    Log.w("Search", "Result count: " + results.getCount());

    for (int i=0; i<results.getCount(); i++) {
        results.getNthDoc(i);
          foundNotes.add(new Note(results.getDocId(), results.getDocDisplayName(), "DOC TEXT", results.getDocDate(), results.getDocDate() ));
    }

    return foundNotes;

}

This method creates and returns a list of notes by iterating over the SearchResults submitted and calling the various getter methods to retrieve the index properties that were stored in our NoteDataSource. Notice that I return "DOC TEXT" as the content of the note. This is a shortcut because I am not going to display the actual text of the note in my ListView. Instead, I’m just showing the title and keeping the ID available so that I can navigate to the full text of the note later.

Finally, I bind the list of notes to my ListView using a standard ArrayAdapter in my BindResultsToListView method in my SearchNotes activity class:

private void BindResultsToListView(List<Note> foundNotes) {

    try {
        ArrayAdapter<Note> dataAdapter = new ArrayAdapter<Note>(
                this, android.R.layout.simple_list_item_1, android.R.id.text1, foundNotes
        );
        ListView lv = (ListView) findViewById(R.id.searchNoteList);
        lv.setAdapter(dataAdapter);
    } catch (Exception ex) {
        ex.printStackTrace();
    }

}

In the sample code, I am using a standard Android simple_list_item_1 layout and binding the results of the Note.toString() method to the text1 element inside of the layout.

Summary

It was that simple to add a rich search engine library to my notepad app. I now have a full-power search utility on my phone that doesn’t need to reach out to Google or Bing or some other service to query my data. The NoteDataSource is complete and handles all of the search logic. I can hand search operations to that class and it will process my notes accordingly. Feel free to extract it, configure it to meet your needs and re-use it in your apps. The complete source code of the app, minus the dtSearch libraries, is available to download as part of this article.

More on dtSearch
dtSearch.com
A Search Engine in Your Pocket – Introducing dtSearch on Android
Blazing Fast Source Code Search in the Cloud
Using Azure Files, RemoteApp and dtSearch for Secure Instant Search Across Terabytes of A Wide Range of Data Types from Any Computer or Device
Windows Azure SQL Database Development with the dtSearch Engine
Faceted Search with dtSearch – Not Your Average Search Filter
Turbo Charge your Search Experience with dtSearch and Telerik UI for ASP.NET
Put a Search Engine in Your Windows 10 Universal (UWP) Applications
Indexing SharePoint Site Collections Using the dtSearch Engine DataSource API
Working with the dtSearch® ASP.NET Core WebDemo Sample Application
Using dtSearch on Amazon Web Services with EC2 & EBS
Full-Text Search with dtSearch and AWS Aurora