Retrieving a file via. HTTP

• Download source files - 3.60 Kb
• Download demo project - 22.1 Kb

In my search to download files using HTTP, I have encountered many good implementations and examples but none met my criteria - I began implementing my own. I needed fetching a file via HTTP and writing this file to disk, displaying the various HTTP codes, in-order to react or display some sort of message to the user. The best solution I found was to derive a class from CInternetSession which allowed me status callbacks and plenty of room to add more functionality to the class as I needed. I'll attempt to walk through this showing how to download files via HTTP.

The steps:

First, we must create an HTTP session using the CInternetSession class as follows.

CInternetSession session(pstrAgent, dwContext, dwAccessType, 
      pstrProxyName, pstrProxyBypass, dwFlags);

The first parameter is a pointer to the string referring to the application name identifying who or what is making the request. The default is NULL that causes the framework to call a global function AfxGetAppName that returns the application name. Following is a DWORD dwContext associated with the identifier of the operation that identifies the status information returned by CInternetSession::OnStatusCallback; the default is 1. If dwFlags includes INTERNET_FLAG_ASYNC, then all objects created by CInternetSession will have asynchronous behavior, but a status callback routine must be registered (explanation follows later.). Next, dwAccessType describes the type of access to be used and only one of the following can be used,

• INTERNET_OPEN_TYPE_PRECONFIG - Gets the preconfigured information from the registry that happens to be the default.
• INTERNET_OPEN_TYPE_DIRECT - Direct to the internet.
• INTERNET_OPEN_TYPE_PROXY - Use a proxy to access the Internet.

pstrProxyName is a pointer to a string referring the name of the proxy to use if dwAccessType is set to INTERNET_OPEN_TYPE_PROXY. The default is NULL. pstrProxyBypass is a pointer to a string referring to a list of optional server addresses if NULL. The list is fetched from the registry. Just as pstrProxyName, INTERNET_OPEN_TYPE_PROXY must be specified for dwAccessType. DWORD dwFlags is to set the options of caching and asynchronous behavior. The default is 0. This can contain any of the following flags,

• INTERNET_FLAG_DONT_CACHE - Do not cache anything.
• INTERNET_FLAG_ASYNC - Enables asynchronous operations as long as a status callback is registered using the function EnableStatusCallback.
• INTERNET_FLAG_OFFLINE - Use the Internet cache folder, if the requested data isn't found then corresponding error is returned.

Then we can set the various options.

session.SetOption(INTERNET_OPTION_CONNECT_TIMEOUT, 1,000 * 20);
session.SetOption(INTERNET_OPTION_CONNECT_BACKOFF, 1000);
session.SetOption(INTERNET_OPTION_CONNECT_RETRIES, 1);

Look in the wininet.h file to see all the options we can use. Many of them there.

Then we can set the status call back routine if we wish to use callbacks. Either set it to TRUE (enable) or FALSE (disable).

session.EnableStatusCallback(TRUE);

Again, look in your wininet.h file for the various callbacks you can react to in your callback routine.

If we use status call backs, make sure they are thread safe protected in the implementation of our callback routine with,

AFX_MANAGE_STATE(AfxGetAppModuleState( ));

Second, we need to connect to the server using CinternetSession::GetHttpConnection. We will need a pointer to a string as the address of the server we are making the connection to, an unsigned short int of the port number we will use for this connection, a pointer to a string containing the username and a pointer to a string containing a password if we are retrieving any data that is password protected. The last two parameters are NULL as the default.

CHttpConnection* pServer = NULL;   

pServer = session.GetHttpConnection(lpstrServer,
                                    usPort,
                                    pstrUserName,
                                    pstrPassword);

Third, we need to open an HTTP request using CHttpConnection::OpenRequest.

CHttpFile* pFile = NULL;
pFile = pServer->OpenRequest(pstrVerb,
                             strFile,
                             pstrReferer,
                             1,
                             &pstrAcceptTypes,
                             pstrVersion,
                             dwHttpRequestFlags);

OpenRequest needs a pointer to a string containing the verb for this request that can be either GET, POST, HEAD, PUT, LINK, DELETE or UNLINK. If NULL, GET will be used. It also needs a pointer to a string containing the file, executable module, even a search specifier that will be associated with the verb that was just specified. Also required is a pointer to a string containing the address from which this request obtained from, if NULL no HTTP header is specified; a DWORD for the context identifier of the OpenRequest, 1 is the default and this DWORD is associated with the specific ChttpConnection object created by the CInternetSession object; a pointer to a null terminated string containing the content type the client accepts. Specifying NULL tells the server the client only accepts documents of type text/*. AcceptTypes is the equivalent of the CGI variable CONTENT_TYPE. OpenRequest needs a pointer string containing the HTTP version that is used, if NULL, HTTP/1.0 will be used. Last, a DWORD specifying the HTTP request flags to be set. Any of the following can be used.

• INTERNET_FLAG_RELOAD - Forces a download from the server and not from the cache.
• INTERNET_FLAG_DONT_CACHE - Do not add any data returned from the server to the cache.
• INTERNET_FLAG_MAKE_PERSISTENT - Add the returned data from the server as a persistent object in the cache.
• INTERNET_FLAG_SECURE - Use secure transactions via SSL/PCT.
• INTERNET_FLAG_NO_AUTO_REDIRECT - Do not allow redirections to be automatically handled.

Now we can just add the various request headers and send the request, hoping for the best if the connection didn't return an error. To add the various headers we can use,

BOOL AddRequestHeaders( CString& str,
    DWORD dwFlags = HTTP_ADDREQ_FLAG_ADD_IF_NEW );

We can use a null terminated string containing the headers we wish to add.

CString szHeaders = "Accept: audio/x-aiff, audio/basic, audio/midi,
  audio/mpeg, audio/wav, image/jpeg, image/gif, image/jpg, image/png,
  image/mng, image/bmp, text/plain, text/html, text/htm\r\n";

The dwFlags can contain any of the following.

• HTTP_ADDREQ_FLAG_COALESCE - Merges all headers of the same name separated by a comma.
• HTTP_ADDREQ_FLAG_REPLACE - Remove and add to replace the current header.
• HTTP_ADDREQ_FLAG_ADD_IF_NEW - Only adds the header if one doesn't exist.
• HTTP_ADDREQ_FLAG_ADD - Used this with REPLACE and it adds the header if it doesn't exist.

pFile->AddRequestHeaders(szHeaders);
pFile->AddRequestHeaders("User-Agent: GetWebFile/1.0\r\n",
            HTTP_ADDREQ_FLAG_ADD_IF_NEW);

Then just send the request,

pFile->SendRequest();

Then we can query the status of our request that will tell us if the file is there, missing, redirected, etc.

DWORD dwRet;
pFile->QueryInfoStatusCode(dwRet);

Then act accordingly depending on the status code.

if (dwRet == HTTP_STATUS_DENIED)
{
do something
}

if (dwRet == HTTP_STATUS_MOVED ||
    dwRet == HTTP_STATUS_REDIRECT ||
    dwRet == HTTP_STATUS_REDIRECT_METHOD)
{
     do something
}

If everything went well, QueryInfoStatusCode will return HTTP_STATUS_OK. Then we can allocate a buffer, create our file to write data in the buffer to, read the file until the end of file and do any clean up we need to do once we're done. Not forgetting to handle any exceptions we may come across and close our CInterntSession object.

if(dwRet == HTTP_STATUS_OK)
  {
    int len = pFile->GetLength();
    char buf[2000];
    int numread;
    CString filepath;
    filepath.Format(".\\Files\\%s", strFile);
    CFile myfile(filepath,
   CFile::modeCreate|CFile::modeWrite|CFile::typeBinary);
    while ((numread = pFile->Read(buf,sizeof(buf)-1)) > 0)
            {
                 buf[numread] = '\0';
        strFile += buf;
        myfile.Write(buf, numread);
        pWebFileDlg->m_ctrlProgress.StepIt();
        PeekAndPump();
           }
    myfile.Close();
  }
pFile->Close(); 
delete pFile;
pServer->Close();
delete pServer;
session.Close();
pWebFileDlg->m_ctrlProgress.SetPos(0);

To use the class that I provide in the demo, the function to call is

DWORD CMyInternetSession::GetWebFile(LPCTSTR pstrAgent,
        LPCTSTR lpstrServer, int nPort, CString strFile);

GetWebFile will return a DWORD associated with the returned HTTP error code. pstrAgent is a string containing the client name, lpstrServer is a string containing the server address, nPort is an int of the port number to use, strFile is a CString string of the filename to fetch from the server.

Also the following function,

void CMyInternetSession::ShowStatus(LPCTSTR strStatus)
{
    CGetWebFileDlg * pWebFileDlg = (CGetWebFileDlg*) AfxGetMainWnd();
    pWebFileDlg->ShowStatus(strStatus);
}

should handle any displaying of status information in any fashion at the programmer's wish.

TIP: With the platform SDK, a debug version of wininet.dll can be had that comes in very handy in debugging.

Then we're done! Hope this has been of some insight to anyone looking for an example with explanation on the subject. Just include MyInternetSession.cpp and MyInternetSession.h into your project and edit it to your liking. Take a close look at the function ShowStatus. In addition, I used a peek and pump that I borrowed from Chris Maunder within the file reading routine, and use a stock MFC progress control. I would change these to your liking. Many possibilities can be implemented, one could change it to a function within a thread and so on.

If any bugs, ideas or additions, just leave the author an e-mail. Kudos! Constructive criticism is welcomed.

Wish list:

• Implementing a pointer to a status or edit control specified by the programmer.
• Better handling of error control.
• Implementing the HTTP_STATUS_DENIED status return code in a more productive way.

References

• Various information from - MSDN