Click here to Skip to main content
12,401,984 members (50,882 online)
Click here to Skip to main content
Add your own
alternative version

Stats

16.6K views
1.8K downloads
15 bookmarked
Posted

YaCy P2P Search Sample in C#

, 10 Sep 2010 CPOL
Rate this:
Please Sign up or sign in to vote.
A C# sample of how to query a YaCy peer.

Introduction

This is a quick example of how to query a YaCy peer for search results using C#.

Background

YaCy is an Open Source peer to peer web search engine/crawler. It can be used to provide search functionality for local intranets and document stores, or it can contribute to a global P2P network to provide web crawling/indexing/search in a decentralized fashion.

Read more at http://yacy.net.

Using the Code

The YacyServer class (below) is a simple class that keeps track of the URI of the YaCy peer you wish to use and details of your web proxy (if needed).

The URI passed to the constructor should be an active YaCy peer.

The main method used is Search, which has an override to just use the default maximum number of results. As you can see, it's just a web request, and the results are RSS.

The return result is an enumeration of SearchResult objects.

/// <summary>
/// Encapsulates a YaCy web server
/// </summary>
public class YacyServer
{
    private Uri uri;
    private IWebProxy proxy;

    public YacyServer(Uri webServerUri) : this(null, webServerUri) { }

    public YacyServer(IWebProxy proxy, Uri webServerUri)
    {
        uri = webServerUri;
        this.proxy = proxy;            
    }

    private int _max = 500;
    public int MaxResultDefault
    {
        get { return _max; }
        set { if (value != 0) _max = Math.Abs(value); }
    }

    public IEnumerable<SearchResult> Search(string search)
    {
        return Search(0, search);
    }

    public IEnumerable<SearchResult> Search(int maxResults, string search)
    {
        List<SearchResult> result = new List<SearchResult>();

        string request = uri.Scheme + "://" + uri.Host + ":" + uri.Port + 
                         "/yacysearch.rss?query=" + HttpUtility.UrlEncode(search);
        if(maxResults > 0)
            request += "&maximumRecords=" + maxResults.ToString();
        else
            request += "&maximumRecords=" + _max.ToString();

        request += "&resource=global";
        // important, don't just search the local index

        WebRequest wr = (WebRequest)HttpWebRequest.Create(request);
        if (proxy != null)
            wr.Proxy = proxy;

        WebResponse resp = wr.GetResponse();
        XmlDocument doc = new XmlDocument();
        doc.Load(resp.GetResponseStream());

        XmlNodeList results = doc.SelectNodes("/rss/channel/item");

        foreach (XmlNode node in results)
            result.Add(new SearchResult(node));

        return (IEnumerable<SearchResult>)result;
    }
}

Obviously, there are a number of things not shown: error handling, or making the web request asynchronous, for example.

The SearchResult class is just a wrapper:

/// <summary>
/// Encapsulates a YaCy search result
/// </summary>
public class SearchResult
{
    private XmlNode data;

    protected internal SearchResult(XmlNode data)
    {
        this.data = data;
    }

    public string Title
    {
        get { return data["title"].InnerText; }
    }

    public string Link
    {
        get { return data["link"].InnerText; }
    }

    public string Description
    {
        get { return data["description"].InnerText; }
    }
}

Using these classes in a Windows Form goes something like this:

private void Form1_Load(object sender, EventArgs e)
{
    // this is my yacy peer address,
    // read how to set up your own yacy peer at http://yacy.net
    server = new YacyServer(new Uri("http://abbot.is-a-geek.net:8080"));
}

private YacyServer server;
private void button1_Click(object sender, EventArgs e)
{
    button1.Enabled = false;
    IEnumerable<SearchResult> result = server.Search(textBox1.Text);

    // The search result uses this interface
    // so it can be further filtered using Linq
    // eg:
    // var x = from p in result where p.Title.Contains("Blog") select p;

    dataGridView1.DataSource = result;

    button1.Enabled = true;
}

Points of Interest

The Search function returns IEnumerable<T>, so the search results could be filtered further on the client using LINQ.

For further API documentation, click here.

Summary

YaCy has a large API and can search many types of content; if anyone wants to request further sample YaCy articles, I will do my best to oblige.

YaCy offers an Open Source, peer to peer search solution which can be used in many ways to improve your own website as well as the freedom of the internet (YaCy can't be censored as there is no central point of control).

The public peer network ('freeworld') is still growing, please get involved.

License

This article, along with any associated source code and files, is licensed under The Code Project Open License (CPOL)

Share

About the Author

Nathan St
Web Developer
United Kingdom United Kingdom
I've been a web application developer since the late 1990s, working for eCommerce, media and telecoms companies across europe and america.

You may also be interested in...

Comments and Discussions

 
GeneralSmall but effective Pin
Hiren Solanki10-Sep-10 19:50
memberHiren Solanki10-Sep-10 19:50 
GeneralMy vote of 3 Pin
anishmehta18-Aug-10 7:00
memberanishmehta18-Aug-10 7:00 
GeneralMy vote of 5 Pin
yacy16-Aug-10 13:43
memberyacy16-Aug-10 13:43 

General General    News News    Suggestion Suggestion    Question Question    Bug Bug    Answer Answer    Joke Joke    Praise Praise    Rant Rant    Admin Admin   

Use Ctrl+Left/Right to switch messages, Ctrl+Up/Down to switch threads, Ctrl+Shift+Left/Right to switch pages.

| Advertise | Privacy | Terms of Use | Mobile
Web02 | 2.8.160721.1 | Last Updated 10 Sep 2010
Article Copyright 2010 by Nathan St
Everything else Copyright © CodeProject, 1999-2016
Layout: fixed | fluid