Click here to Skip to main content
11,712,820 members (90,311 online)
Click here to Skip to main content

Retrieving a file via. HTTP

, 24 Feb 2002 234.3K 8.8K 82
Rate this:
Please Sign up or sign in to vote.
A tutorial and class derived from CInternetSession.

Sample Image - GetWebFile.gif

In my search to download files using HTTP, I have encountered many good implementations and examples but none met my criteria - I began implementing my own. I needed fetching a file via HTTP and writing this file to disk, displaying the various HTTP codes, in-order to react or display some sort of message to the user. The best solution I found was to derive a class from CInternetSession which allowed me status callbacks and plenty of room to add more functionality to the class as I needed. I'll attempt to walk through this showing how to download files via HTTP.

The steps:

First, we must create an HTTP session using the CInternetSession class as follows.

CInternetSession session(pstrAgent, dwContext, dwAccessType, 
      pstrProxyName, pstrProxyBypass, dwFlags);

The first parameter is a pointer to the string referring to the application name identifying who or what is making the request. The default is NULL that causes the framework to call a global function AfxGetAppName that returns the application name. Following is a DWORD dwContext associated with the identifier of the operation that identifies the status information returned by CInternetSession::OnStatusCallback; the default is 1. If dwFlags includes INTERNET_FLAG_ASYNC, then all objects created by CInternetSession will have asynchronous behavior, but a status callback routine must be registered (explanation follows later.). Next, dwAccessType describes the type of access to be used and only one of the following can be used,

  • INTERNET_OPEN_TYPE_PRECONFIG - Gets the preconfigured information from the registry that happens to be the default.
  • INTERNET_OPEN_TYPE_DIRECT - Direct to the internet.
  • INTERNET_OPEN_TYPE_PROXY - Use a proxy to access the Internet.

pstrProxyName is a pointer to a string referring the name of the proxy to use if dwAccessType is set to INTERNET_OPEN_TYPE_PROXY. The default is NULL. pstrProxyBypass is a pointer to a string referring to a list of optional server addresses if NULL. The list is fetched from the registry. Just as pstrProxyName, INTERNET_OPEN_TYPE_PROXY must be specified for dwAccessType. DWORD dwFlags is to set the options of caching and asynchronous behavior. The default is 0. This can contain any of the following flags,

  • INTERNET_FLAG_DONT_CACHE - Do not cache anything.
  • INTERNET_FLAG_ASYNC - Enables asynchronous operations as long as a status callback is registered using the function EnableStatusCallback.
  • INTERNET_FLAG_OFFLINE - Use the Internet cache folder, if the requested data isn't found then corresponding error is returned.

Then we can set the various options.

session.SetOption(INTERNET_OPTION_CONNECT_TIMEOUT, 1,000 * 20);
session.SetOption(INTERNET_OPTION_CONNECT_BACKOFF, 1000);
session.SetOption(INTERNET_OPTION_CONNECT_RETRIES, 1);

Look in the wininet.h file to see all the options we can use. Many of them there.

Then we can set the status call back routine if we wish to use callbacks. Either set it to TRUE (enable) or FALSE (disable).

session.EnableStatusCallback(TRUE);

Again, look in your wininet.h file for the various callbacks you can react to in your callback routine.

If we use status call backs, make sure they are thread safe protected in the implementation of our callback routine with,

AFX_MANAGE_STATE(AfxGetAppModuleState( ));

Second, we need to connect to the server using CinternetSession::GetHttpConnection. We will need a pointer to a string as the address of the server we are making the connection to, an unsigned short int of the port number we will use for this connection, a pointer to a string containing the username and a pointer to a string containing a password if we are retrieving any data that is password protected. The last two parameters are NULL as the default.

CHttpConnection* pServer = NULL;   

pServer = session.GetHttpConnection(lpstrServer,
                                    usPort,
                                    pstrUserName,
                                    pstrPassword);

Third, we need to open an HTTP request using CHttpConnection::OpenRequest.

CHttpFile* pFile = NULL;
pFile = pServer->OpenRequest(pstrVerb,
                             strFile,
                             pstrReferer,
                             1,
                             &pstrAcceptTypes,
                             pstrVersion,
                             dwHttpRequestFlags);

OpenRequest needs a pointer to a string containing the verb for this request that can be either GET, POST, HEAD, PUT, LINK, DELETE or UNLINK. If NULL, GET will be used. It also needs a pointer to a string containing the file, executable module, even a search specifier that will be associated with the verb that was just specified. Also required is a pointer to a string containing the address from which this request obtained from, if NULL no HTTP header is specified; a DWORD for the context identifier of the OpenRequest, 1 is the default and this DWORD is associated with the specific ChttpConnection object created by the CInternetSession object; a pointer to a null terminated string containing the content type the client accepts. Specifying NULL tells the server the client only accepts documents of type text/*. AcceptTypes is the equivalent of the CGI variable CONTENT_TYPE. OpenRequest needs a pointer string containing the HTTP version that is used, if NULL, HTTP/1.0 will be used. Last, a DWORD specifying the HTTP request flags to be set. Any of the following can be used.

  • INTERNET_FLAG_RELOAD - Forces a download from the server and not from the cache.
  • INTERNET_FLAG_DONT_CACHE - Do not add any data returned from the server to the cache.
  • INTERNET_FLAG_MAKE_PERSISTENT - Add the returned data from the server as a persistent object in the cache.
  • INTERNET_FLAG_SECURE - Use secure transactions via SSL/PCT.
  • INTERNET_FLAG_NO_AUTO_REDIRECT - Do not allow redirections to be automatically handled.

Now we can just add the various request headers and send the request, hoping for the best if the connection didn't return an error. To add the various headers we can use,

BOOL AddRequestHeaders( CString& str,
    DWORD dwFlags = HTTP_ADDREQ_FLAG_ADD_IF_NEW );

We can use a null terminated string containing the headers we wish to add.

CString szHeaders = "Accept: audio/x-aiff, audio/basic, audio/midi,
  audio/mpeg, audio/wav, image/jpeg, image/gif, image/jpg, image/png,
  image/mng, image/bmp, text/plain, text/html, text/htm\r\n";

The dwFlags can contain any of the following.

  • HTTP_ADDREQ_FLAG_COALESCE - Merges all headers of the same name separated by a comma.
  • HTTP_ADDREQ_FLAG_REPLACE - Remove and add to replace the current header.
  • HTTP_ADDREQ_FLAG_ADD_IF_NEW - Only adds the header if one doesn't exist.
  • HTTP_ADDREQ_FLAG_ADD - Used this with REPLACE and it adds the header if it doesn't exist.
pFile->AddRequestHeaders(szHeaders);
pFile->AddRequestHeaders("User-Agent: GetWebFile/1.0\r\n",
            HTTP_ADDREQ_FLAG_ADD_IF_NEW);

Then just send the request,

pFile->SendRequest();

Then we can query the status of our request that will tell us if the file is there, missing, redirected, etc.

DWORD dwRet;
pFile->QueryInfoStatusCode(dwRet);

Then act accordingly depending on the status code.

if (dwRet == HTTP_STATUS_DENIED)
{
do something
}

if (dwRet == HTTP_STATUS_MOVED ||
    dwRet == HTTP_STATUS_REDIRECT ||
    dwRet == HTTP_STATUS_REDIRECT_METHOD)
{
     do something
}

If everything went well, QueryInfoStatusCode will return HTTP_STATUS_OK. Then we can allocate a buffer, create our file to write data in the buffer to, read the file until the end of file and do any clean up we need to do once we're done. Not forgetting to handle any exceptions we may come across and close our CInterntSession object.

if(dwRet == HTTP_STATUS_OK)
  {
    int len = pFile->GetLength();
    char buf[2000];
    int numread;
    CString filepath;
    filepath.Format(".\\Files\\%s", strFile);
    CFile myfile(filepath,
   CFile::modeCreate|CFile::modeWrite|CFile::typeBinary);
    while ((numread = pFile->Read(buf,sizeof(buf)-1)) > 0)
            {
                 buf[numread] = '\0';
        strFile += buf;
        myfile.Write(buf, numread);
        pWebFileDlg->m_ctrlProgress.StepIt();
        PeekAndPump();
           }
    myfile.Close();
  }
pFile->Close(); 
delete pFile;
pServer->Close();
delete pServer;
session.Close();
pWebFileDlg->m_ctrlProgress.SetPos(0);

To use the class that I provide in the demo, the function to call is

DWORD CMyInternetSession::GetWebFile(LPCTSTR pstrAgent,
        LPCTSTR lpstrServer, int nPort, CString strFile);

GetWebFile will return a DWORD associated with the returned HTTP error code. pstrAgent is a string containing the client name, lpstrServer is a string containing the server address, nPort is an int of the port number to use, strFile is a CString string of the filename to fetch from the server.

Also the following function,

void CMyInternetSession::ShowStatus(LPCTSTR strStatus)
{
    CGetWebFileDlg * pWebFileDlg = (CGetWebFileDlg*) AfxGetMainWnd();
    pWebFileDlg->ShowStatus(strStatus);
}

should handle any displaying of status information in any fashion at the programmer's wish.

TIP: With the platform SDK, a debug version of wininet.dll can be had that comes in very handy in debugging.

Then we're done! Hope this has been of some insight to anyone looking for an example with explanation on the subject. Just include MyInternetSession.cpp and MyInternetSession.h into your project and edit it to your liking. Take a close look at the function ShowStatus. In addition, I used a peek and pump that I borrowed from Chris Maunder within the file reading routine, and use a stock MFC progress control. I would change these to your liking. Many possibilities can be implemented, one could change it to a function within a thread and so on.

If any bugs, ideas or additions, just leave the author an e-mail. Kudos! Constructive criticism is welcomed.

Wish list:

  • Implementing a pointer to a status or edit control specified by the programmer.
  • Better handling of error control.
  • Implementing the HTTP_STATUS_DENIED status return code in a more productive way.

References

  • Various information from - MSDN

License

This article has no explicit license attached to it but may contain usage terms in the article text or the download files themselves. If in doubt please contact the author via the discussion board below.

A list of licenses authors might use can be found here

Share

About the Author

Jay Beckert
Web Developer
United States United States
Started programming with basic in highschool then moved on to C and eventually C++ while obtaining a degree in CS at NFU. Using MFC for the past 3 years when I decided dos programming died.

You may also be interested in...

Comments and Discussions

 
Questioncan anybody tell me how can this code work? Pin
cct828-Aug-14 19:23
membercct828-Aug-14 19:23 
Generalnon admin sendRequest function is failing Pin
satishadimula8-Feb-11 23:51
membersatishadimula8-Feb-11 23:51 
Generalvery good Pin
wanderingman14-Dec-10 11:09
memberwanderingman14-Dec-10 11:09 
GeneralMy vote of 4 Pin
vviicckkyy25-Oct-10 23:18
membervviicckkyy25-Oct-10 23:18 
GeneralMy vote of 1 Pin
oly9991-Feb-10 23:11
memberoly9991-Feb-10 23:11 
GeneralFile is not proper Pin
FaizD29-Sep-08 23:54
memberFaizD29-Sep-08 23:54 
QuestionCan't not find hcw error Pin
angelaflowers19-Apr-08 12:20
memberangelaflowers19-Apr-08 12:20 
GeneralSOAP POST request Pin
AllaB12-Mar-07 9:36
memberAllaB12-Mar-07 9:36 
GeneralC# editor Pin
linlian4-Dec-06 7:29
memberlinlian4-Dec-06 7:29 
QuestionRequest timeout Pin
marlongrech15-Nov-06 11:50
membermarlongrech15-Nov-06 11:50 
GeneralCannot deal with UNICODE Pin
scvarz29-Aug-06 0:13
memberscvarz29-Aug-06 0:13 
QuestionWill this work on a UNIX Box ? Pin
Vikrant for VC++29-May-06 22:59
memberVikrant for VC++29-May-06 22:59 
GeneralMany thanks Pin
cfilorux11-Apr-06 3:53
membercfilorux11-Apr-06 3:53 
QuestionNot Getting All Files Pin
zubair_ahmed10-Jan-06 2:58
memberzubair_ahmed10-Jan-06 2:58 
QuestionDownload large files with POST Pin
TopFire612-Dec-05 2:48
memberTopFire612-Dec-05 2:48 
GeneralApplication crashes if directory doesn't exist Pin
benni6-Oct-05 23:55
memberbenni6-Oct-05 23:55 
QuestionPeekAndPump ??? Pin
Y_R13-Aug-05 2:43
memberY_R13-Aug-05 2:43 
AnswerRe: PeekAndPump ??? Pin
xxyyzzaa18-Aug-05 2:54
memberxxyyzzaa18-Aug-05 2:54 
GeneralRe: PeekAndPump ??? Pin
Y_R18-Aug-05 9:35
memberY_R18-Aug-05 9:35 
GeneralSendRequest() failure Pin
chandugsekhar13-Jun-05 17:32
memberchandugsekhar13-Jun-05 17:32 
AnswerRe: SendRequest() failure Pin
DijaS6-May-07 20:33
memberDijaS6-May-07 20:33 
GeneralWin XP/2003 Pin
Marco Tezza25-May-05 7:02
memberMarco Tezza25-May-05 7:02 
QuestionSpeed up download? Pin
abest96-Apr-05 12:41
memberabest96-Apr-05 12:41 
AnswerRe: Speed up download? Pin
danidanidani27-Jun-05 6:02
memberdanidanidani27-Jun-05 6:02 
GeneralRe: Speed up download? Pin
Y_R13-Aug-05 2:56
memberY_R13-Aug-05 2:56 
GeneralRe: Speed up download? Pin
danidanidani13-Aug-05 7:24
memberdanidanidani13-Aug-05 7:24 
GeneralRe: Speed up download? Pin
fengforky11-Nov-12 4:09
memberfengforky11-Nov-12 4:09 
QuestionGetLength() bug? Pin
CodeHead22-Feb-05 6:24
memberCodeHead22-Feb-05 6:24 
AnswerRe: GetLength() bug? Pin
Y_R20-Aug-05 1:02
memberY_R20-Aug-05 1:02 
GeneralRe: GetLength() bug? Pin
Y_R22-Aug-05 10:44
memberY_R22-Aug-05 10:44 
GeneralRe: GetLength() bug? Pin
Abris30-Mar-06 2:12
memberAbris30-Mar-06 2:12 
GeneralAbout connection Pin
azorea29-Dec-04 11:19
sussazorea29-Dec-04 11:19 
GeneralFailure to Pull From Cache Pin
StanBurton23-Sep-04 15:24
memberStanBurton23-Sep-04 15:24 
GeneralHTTP/1.0 ??? using OpenRequest Pin
derekviljoen14-May-04 4:46
memberderekviljoen14-May-04 4:46 
Questionwhat cud be wrong Pin
faidoo19-Apr-04 20:25
memberfaidoo19-Apr-04 20:25 
GeneralMaking into a DLL Pin
timrmci21-Jan-04 5:36
membertimrmci21-Jan-04 5:36 
GeneralProgress Bar when downloading big files Pin
Braulio Díez20-Jan-04 20:52
memberBraulio Díez20-Jan-04 20:52 
GeneralRe: Progress Bar when downloading big files Pin
Terry O'Nolley19-May-04 17:42
memberTerry O'Nolley19-May-04 17:42 
Generalgood one Pin
Balkrishna Talele6-Jan-04 20:43
memberBalkrishna Talele6-Jan-04 20:43 
GeneralAbout inserting classes... Pin
Almenara2-Jan-04 8:51
memberAlmenara2-Jan-04 8:51 
GeneralHTML download and parse Pin
mattroos4-Dec-03 13:37
membermattroos4-Dec-03 13:37 
Generalnot working for URL with subdirectory file path Pin
javajunki13-Nov-03 6:03
sussjavajunki13-Nov-03 6:03 
GeneralRe: not working for URL with subdirectory file path Pin
Almenara13-Jan-04 19:49
memberAlmenara13-Jan-04 19:49 
GeneralRe: not working for URL with subdirectory file path Pin
Basava12325-Feb-04 2:23
memberBasava12325-Feb-04 2:23 
AnswerRe: not working for URL with subdirectory file path Pin
DijaS6-May-07 20:17
memberDijaS6-May-07 20:17 
QuestionHTTPS? Pin
Ed K16-Oct-03 11:57
memberEd K16-Oct-03 11:57 
QuestionBug on CInternetSession::SetOption??? Pin
naghicalin16-Oct-03 11:34
membernaghicalin16-Oct-03 11:34 
Questionwhile the network is interrupted,can it support download at the last position after the network return to normal? Pin
softarts26-Feb-03 13:49
membersoftarts26-Feb-03 13:49 
GeneralAsynchronous support Pin
Jai b5-Feb-03 2:51
memberJai b5-Feb-03 2:51 
GeneralRe: Asynchronous support Pin
Jay Beckert7-Feb-03 14:10
memberJay Beckert7-Feb-03 14:10 

General General    News News    Suggestion Suggestion    Question Question    Bug Bug    Answer Answer    Joke Joke    Rant Rant    Admin Admin   

Use Ctrl+Left/Right to switch messages, Ctrl+Up/Down to switch threads, Ctrl+Shift+Left/Right to switch pages.

| Advertise | Privacy | Terms of Use | Mobile
Web04 | 2.8.150819.1 | Last Updated 25 Feb 2002
Article Copyright 2002 by Jay Beckert
Everything else Copyright © CodeProject, 1999-2015
Layout: fixed | fluid