Code Project Forums : New Posts Monitor

• Download demo with source - 219 KB

Introduction

I wrote this application because I wanted to get notified whenever there was a new post in specific forums. In particular I was focusing on the General Indian Topics forum, because Chris had asked me to moderate it and help develop it into an orientation forum for newbie Indian CPians. I found that I missed out on new threads or when someone replied to someone else (and thus I didn't get an email). And obviously it became a little tedious to keep refreshing the page every 3-4 minutes. That's when I thought it'd be simpler to write a small app that would pull the HTML for that forum and compare the forum count with the previous pull. And as I started on it I thought I'd just do it for all forums, and the easiest way to do this was by pulling the HTML for the main forum listing.

The application was written using VS 2010/.NET 4.0 so you'll need the .NET 4.0 runtime. I don't use any .NET 4.0 features so theoretically you can run it on 3.5 if you rebuild the project with that target framework or if you downgrade to VS 2008. It should be fairly trivial to do as it's a very simple project. The project uses the HTML Agility Pack v1.4.0 from CodePlex, so you'll need that too. I have included the required DLL in the article download, so that should suffice. Here's the URL for the HTML Agility Pack, including source downloads and documentation.

• HTML Agility Pack (CodePlex)

Using the application

The application has a fairly simple UI. When you run it, the frequency slider will default to 30 seconds. What it means is that the app will fetch the HTML every 30 seconds. Unless you are expecting to reply live, you might want to lower that to 180 seconds or so. The largest setting you can set is 300 seconds (5 minutes). I reckoned that for anything above 5 minutes, you don't really need this app. CP's sure to have had new posts in most forums in any 5 minute span of time (since it's got active members from all over the world and thus different time zones). There is a button that will perform a manual fetch. This will not affect the running timer though. If you are getting up to go get some coffee, you could do a manual fetch and that way you know that when you come back, whatever you see is what's been posted in your absence. I've personally used it more for debugging really than for anything else. Anyway, whenever a forum has new posts, it'll show up (see Figure 1 above). If you set it to 30 seconds, you will often find that there have been no new posts (in any of the forums). Yeah, even CP's not popular enough for that :-)

Remember that the forum counts are based on delta values. So if you set it to run every 1 minute, and you take a 3 minute break, you'll only see the new posts in the last 1 minute. Any new posts  that came up in the previous 2 minutes will have flashed on screen and gone away by the time you return. Sometimes you would actually want to know what you missed. That's what the history window is for - see Figure 2 above. It'll show you the last 500 updates that triggered.

Note how I've tried to use greenish and orange colors for the UI, this was intentionally and painstakingly done to give the app a CP like theme. If you don't like it, then all I can say is that you have appalling eyesight and absolutely no visual taste at all. Sucks to be you! *grin*

Implementation details

Like everyone else who writes Windows apps in the year 2010, I too used WPF/MVVM. In fact I would be terribly shocked if anyone told me that you can write non-WPF apps these days. Alright, I am only kidding here :-)

There is a CodeProjectForum class that represents a forum (along with its delta count).

class CodeProjectForum : INotifyPropertyChanged
{
    private string name;

    public string Name
    {
        get
        {
            return this.name;
        }

        set
        {
            if (this.name != value)
            {
                this.name = value;
                FirePropertyChanged("Name");
            }
        }
    }

    private string description;

    public string Description
    {
        get
        {
            return this.description;
        }

        set
        {
            if (this.description != value)
            {
                this.description = value;
                FirePropertyChanged("Description");
            }
        }
    }

    private int postCount = -1;

    public int PostCount
    {
        get
        {
            return this.postCount;
        }

        set
        {
            this.PreviousPostCount = this.postCount == -1 ? value : this.postCount;
            this.postCount = value;
            FirePropertyChanged("PostCount");
            FirePropertyChanged("Delta");
        }
    }

    public int Delta
    {
        get
        {
            return this.PostCount - this.PreviousPostCount;
        }
    }

    private int previousPostCount;

    public int PreviousPostCount
    {
        get
        {
            return this.previousPostCount;
        }

        private set
        {
            if (this.previousPostCount != value)
            {
                this.previousPostCount = value;
                FirePropertyChanged("PreviousPostCount");
            }
        }        
    }

    private DateTime lastChecked;

    public DateTime LastChecked
    {
        get
        {
            return this.lastChecked;
        }

        set
        {
            if (this.lastChecked != value)
            {
                this.lastChecked = value;
                FirePropertyChanged("LastChecked");
            }
        }
    }

    public event PropertyChangedEventHandler PropertyChanged;

    protected void FirePropertyChanged(string propertyName)
    {
        PropertyChangedEventHandler handler = PropertyChanged;

        if (handler != null)
        {
            handler(this, new PropertyChangedEventArgs(propertyName));
        }
    }

    public CodeProjectForum Clone()
    {
        return new CodeProjectForum()
        {
            Name = this.Name,
            Description = this.Description,
            PostCount = this.PostCount,
            PreviousPostCount  = this.PreviousPostCount,
            LastChecked = this.LastChecked
        };
    }
}

And here's the class that fetches the HTML, parses it, and extracts the forum details. It uses a background worker to fetch and parse the HTML, and when it's done doing that it fires the FetchCompleted event (handled in the UI by the View Model class). The HTML parsing is nothing major, and is also further simplified by using the HtmlAgilityPack.

class CodeProjectForumCountFetcher
{
    private List<CodeProjectForum> forums = new List<CodeProjectForum>();

    public event EventHandler<CodeProjectForumCountFetcherEventArgs> FetchCompleted;

    public void Fetch()
    {
        BackgroundWorker worker = new BackgroundWorker();
        worker.DoWork += Worker_DoWork;
        worker.RunWorkerCompleted += Worker_RunWorkerCompleted;
        worker.RunWorkerAsync();
    }

    private void Worker_RunWorkerCompleted(object sender, RunWorkerCompletedEventArgs e)
    {
        this.FireFetchCompleted();
    }

    private void Worker_DoWork(object sender, DoWorkEventArgs e)
    {
        string html = GetHttpPage(
          "http://www.codeproject.com/script/Forums/List.aspx", 10000);

        HtmlDocument document = new HtmlDocument();
        document.LoadHtml(html);

        foreach (HtmlNode trNode in 
          document.DocumentNode.SelectNodes("//table[@id='ForumListTable']/tr"))
        {
            var tdNodes = trNode.SelectNodes("td");

            if (tdNodes.Count != 4)
                continue;

            var forumName = tdNodes[0].InnerText.Trim();
            if (forumName.ToLowerInvariant().StartsWith("forum"))
                continue;

            int forumCount = -1;
            Int32.TryParse(tdNodes[3].InnerText.Replace(",", String.Empty).Trim(), 
                                                        out forumCount);

            var forumDescription = tdNodes[1].InnerText.Replace(" ", String.Empty).Trim();

            var forumMatches = forums.Where(forum => forum.Name == forumName);
            if (forumMatches.Count() == 0)
            {
                forums.Add(new CodeProjectForum() { Name = forumName, 
                  Description = forumDescription, 
                  PostCount = forumCount, LastChecked = DateTime.Now });
            }
            else
            {
                forumMatches.First().PostCount = forumCount;
                forumMatches.First().LastChecked = DateTime.Now;
            }
        }
    }

    private string GetHttpPage(string url, int timeout)
    {
        var request = WebRequest.Create(new Uri(url, UriKind.Absolute));
        request.Timeout = timeout;
        using (var response = request.GetResponse())
        {
            using (var responseStream = response.GetResponseStream())
            {
                using (var reader = new StreamReader(responseStream))
                {
                    return reader.ReadToEnd();
                }
            }
        }
    }

    public void FireFetchCompleted()
    {
        if (this.FetchCompleted != null)
        {
            this.FetchCompleted(this, new CodeProjectForumCountFetcherEventArgs() 
              { FetchedForums = Array.AsReadOnly(forums.Select(f => f.Clone()).ToArray()) });
        }
    }
}

Both the main window and the history window use a ListBox to display the forum details. They are just styled very differently. There's only one View Model, for the history window I directly use an ObservableCollection<> as its DataContext. Here're some snippets from the View Model class. You can browse the complete source code from the attached article download.

internal class MainWindowViewModel : ViewModelBase
{
    // . . .
    
    public int FetchFrequency
    {
        get
        {
            return this.fetchFrequency;
        }

        set
        {
            if (fetchFrequency != value)
            {
                fetchFrequency = value;
                FirePropertyChanged("FetchFrequency");
            }
        }            
    }

    private string statusText;

    public string StatusText
    {
        get
        {
            return this.statusText;
        }

        set
        {
            if (statusText != value)
            {
                statusText = value;
                FirePropertyChanged("StatusText");
            }
        }
    }

    public MainWindowViewModel()
    {
        this.Forums = new ObservableCollection<CodeProjectForum>();
        this.PropertyChanged += MainWindowViewModel_PropertyChanged;

        fetcher.FetchCompleted += Fetcher_FetchCompleted;
        Fetch();

        timer.Tick += Timer_Tick;
        StartTimer();
    }

    private void StartTimer()
    {
        timer.Interval = new TimeSpan(0, 0, this.FetchFrequency);
        timer.Start();
        ResetStatusText();
    }

    private void ResetStatusText()
    {
        this.StatusText = String.Format("Timer running at {0} seconds.", 
                                        this.FetchFrequency);
    }

    void MainWindowViewModel_PropertyChanged(object sender, PropertyChangedEventArgs e)
    {
        if (e.PropertyName == "FetchFrequency")
        {
            timer.Stop();
            StartTimer();
        }
    }

    private void Fetch()
    {
        this.StatusText = String.Format("Executing a fetch!");
        fetcher.Fetch();
    }

The below code starts with the ShowHistory method where you can see how the HistoryWindow is created if it's not already shown, and how its DataContext is directly set to the collection of forum objects that were previously added.

    private void ShowHistory()
    {
        if(historyWindow == null)
        {
            historyWindow = new HistoryWindow() 
              { Owner = App.Current.MainWindow, DataContext = this.forumHistory };
            historyWindow.Closed += (s, e) => { historyWindow = null; };
            historyWindow.Show();
        }
    }

    void Timer_Tick(object sender, EventArgs e)
    {
        Fetch();
    }

    private void UpdateForumsCollection(IEnumerable<CodeProjectForum> forums)
    {
        this.Forums.Clear();

        foreach (var item in forums)
        {
            this.Forums.Add(item);
            AddToHistory(item);
        }
    }

    private void AddToHistory(CodeProjectForum item)
    {
        if(forumHistory.Count > 499)
        {
            forumHistory.RemoveAt(499);
        }

        forumHistory.Insert(0, item);
    }

    void Fetcher_FetchCompleted(object sender, CodeProjectForumCountFetcherEventArgs e)
    {
        var updatedForums = e.FetchedForums.Where(
          f => f.Delta > 0).OrderByDescending(item => item.Delta);
        UpdateForumsCollection(updatedForums);
        ResetStatusText();

        if (updatedForums.Count() > 0)
        {
            SystemSounds.Exclamation.Play();
        }
    }
}

That's it! As always, please throw in your comments and feedback, even if it's something really minor. Also feel free to ask for any features although I can't promise to implement them or if I do, then to do it in a specific timeframe.

History

• 9/20/2010 - Article published on The Code Project.