Click here to Skip to main content
Click here to Skip to main content

YaCy P2P Search Sample in C#

By , 10 Sep 2010
Rate this:
Please Sign up or sign in to vote.

Introduction

This is a quick example of how to query a YaCy peer for search results using C#.

Background

YaCy is an Open Source peer to peer web search engine/crawler. It can be used to provide search functionality for local intranets and document stores, or it can contribute to a global P2P network to provide web crawling/indexing/search in a decentralized fashion.

Read more at http://yacy.net.

Using the Code

The YacyServer class (below) is a simple class that keeps track of the URI of the YaCy peer you wish to use and details of your web proxy (if needed).

The URI passed to the constructor should be an active YaCy peer.

The main method used is Search, which has an override to just use the default maximum number of results. As you can see, it's just a web request, and the results are RSS.

The return result is an enumeration of SearchResult objects.

/// <summary>
/// Encapsulates a YaCy web server
/// </summary>
public class YacyServer
{
    private Uri uri;
    private IWebProxy proxy;

    public YacyServer(Uri webServerUri) : this(null, webServerUri) { }

    public YacyServer(IWebProxy proxy, Uri webServerUri)
    {
        uri = webServerUri;
        this.proxy = proxy;            
    }

    private int _max = 500;
    public int MaxResultDefault
    {
        get { return _max; }
        set { if (value != 0) _max = Math.Abs(value); }
    }

    public IEnumerable<SearchResult> Search(string search)
    {
        return Search(0, search);
    }

    public IEnumerable<SearchResult> Search(int maxResults, string search)
    {
        List<SearchResult> result = new List<SearchResult>();

        string request = uri.Scheme + "://" + uri.Host + ":" + uri.Port + 
                         "/yacysearch.rss?query=" + HttpUtility.UrlEncode(search);
        if(maxResults > 0)
            request += "&maximumRecords=" + maxResults.ToString();
        else
            request += "&maximumRecords=" + _max.ToString();

        request += "&resource=global";
        // important, don't just search the local index

        WebRequest wr = (WebRequest)HttpWebRequest.Create(request);
        if (proxy != null)
            wr.Proxy = proxy;

        WebResponse resp = wr.GetResponse();
        XmlDocument doc = new XmlDocument();
        doc.Load(resp.GetResponseStream());

        XmlNodeList results = doc.SelectNodes("/rss/channel/item");

        foreach (XmlNode node in results)
            result.Add(new SearchResult(node));

        return (IEnumerable<SearchResult>)result;
    }
}

Obviously, there are a number of things not shown: error handling, or making the web request asynchronous, for example.

The SearchResult class is just a wrapper:

/// <summary>
/// Encapsulates a YaCy search result
/// </summary>
public class SearchResult
{
    private XmlNode data;

    protected internal SearchResult(XmlNode data)
    {
        this.data = data;
    }

    public string Title
    {
        get { return data["title"].InnerText; }
    }

    public string Link
    {
        get { return data["link"].InnerText; }
    }

    public string Description
    {
        get { return data["description"].InnerText; }
    }
}

Using these classes in a Windows Form goes something like this:

private void Form1_Load(object sender, EventArgs e)
{
    // this is my yacy peer address,
    // read how to set up your own yacy peer at http://yacy.net
    server = new YacyServer(new Uri("http://abbot.is-a-geek.net:8080"));
}

private YacyServer server;
private void button1_Click(object sender, EventArgs e)
{
    button1.Enabled = false;
    IEnumerable<SearchResult> result = server.Search(textBox1.Text);

    // The search result uses this interface
    // so it can be further filtered using Linq
    // eg:
    // var x = from p in result where p.Title.Contains("Blog") select p;

    dataGridView1.DataSource = result;

    button1.Enabled = true;
}

Points of Interest

The Search function returns IEnumerable<T>, so the search results could be filtered further on the client using LINQ.

For further API documentation, click here.

Summary

YaCy has a large API and can search many types of content; if anyone wants to request further sample YaCy articles, I will do my best to oblige.

YaCy offers an Open Source, peer to peer search solution which can be used in many ways to improve your own website as well as the freedom of the internet (YaCy can't be censored as there is no central point of control).

The public peer network ('freeworld') is still growing, please get involved.

License

This article, along with any associated source code and files, is licensed under The Code Project Open License (CPOL)

About the Author

Nathan Staudt
Web Developer
United Kingdom United Kingdom
I've been a web application developer since the late 1990s, working for eCommerce, media and telecoms companies across europe and america.
 
I have a BSc (Hons) in computing from Middlesex University and took a few MCP exams several years ago too.
 
Born in America, I grew up in Ireland. My father encouraged my interest in computers as a child when he would bring home his "portable" ibm pc... (20kg or something like that) and I first wrote some stuff in QBasic aged seven or eight, but my real interest in computers took off when my family got internet access at home in the early 90s.
 
Feel free to email me Smile | :)

Comments and Discussions

 
GeneralSmall but effective PinmemberHiren Solanki10-Sep-10 19:50 
GeneralMy vote of 3 Pinmemberanishmehta18-Aug-10 7:00 
GeneralMy vote of 5 Pinmemberyacy16-Aug-10 13:43 

General General    News News    Suggestion Suggestion    Question Question    Bug Bug    Answer Answer    Joke Joke    Rant Rant    Admin Admin   

Use Ctrl+Left/Right to switch messages, Ctrl+Up/Down to switch threads, Ctrl+Shift+Left/Right to switch pages.

| Advertise | Privacy | Mobile
Web03 | 2.8.140415.2 | Last Updated 10 Sep 2010
Article Copyright 2010 by Nathan Staudt
Everything else Copyright © CodeProject, 1999-2014
Terms of Use
Layout: fixed | fluid