Web scraping with progress bar

See more:

ASP.NET

MVC

Originally I planned to build a program as a desktop application but as I thought about it and discussed it with friends we all agreed it would be better served as a website (for the end users). I am familiar with C# but I am completely new to web development.

My current console application for originally building the DB uses this method for web scraping (using HtmlAgilityPack)

public static List<Set> GetSets()
        {
            var sets = new List<Set>();
            var setsDoc = Load("URL");
            var count = SetCount();
            var on = 0;
            foreach (var link in setsDoc.DocumentNode.SelectNodes("//a[@href]"))
            {
                var att = link.Attributes["href"];
                if (att.Value.Contains("URL"))
                {
                    var setName = WebUtility.HtmlDecode(link.FirstChild.InnerText);
                    var setLink = "URL" + att.Value;
                    Console.Clear();
                    Console.WriteLine("Gathering Set data.");
                    Console.WriteLine("Set: " + setName);
                    Console.WriteLine(++on + " out of " + count);
                    DrawProgressBar(on, count, 20, '+');
                    if (!Program.SetExists(setName, setLink))
                    {
                        sets.Add(GetSet(setLink, setName));
                    }
                }
            }
            return sets;
        }

While trying to turn this into a webpage in MVC 5 I am at a complete loss how I could do this, and maintain the progress bar. Normally this action would be very fast and not need a progress bar however the site I am scraping requires that I only hit their website about 1 time every 10 seconds which turns it into something that can take a while.

I am open to any type of solution really, however I want to understand why I would use it and how it works (if not self evident), even some pseudo code or specifics to research would be great. This would be an incredible help and very much appreciated as I am at a loss on where to even get started.