Introduction
Several recent articles have lead me to believe a beginners intro to what is included in
a basic HTTP call is justified. Here is my explanation of the basic parts of both
the text line seen and the actual messages that are sent, along with some sample code
to experiment.
The sample(s) is provided to give one a starting point to look through. The code
submitted is extracted from what I use to test SOAP calls to a service. It
utilizes the MFC CInternetSession class.
The Server side code in the article (not in the down load) is generic C code.
The server code has been run with no modifications as a CGI application under both the
Apache Server (1.3.19) and MS's IIS server.
URL's and the Basic Segments to make a Connection.
Before we get into the code, let's spend a little time getting on the same page.
We must understand what is used by the client to make a server connection and
what is actually sent. The following line was typed into my browser:
http://localhost:680/cgi/cgimfc1?MyInfo=data1+AndMoreInfo=data2
Following completion of the server handling the request my browser displayed this:
Hello Me
Your First Line
This is more text There are 1 Arguments here
The Query String is MyInfo=data1 AndMoreInfo=data2
The Server Soft is Apache/1.3.19 (Win32)
The SERVER_PORT is 680
The REQUEST_METHOD is GET
The REMOTE_USER is
The CONTENT_TYPE is
The CONTENT_LENGTH is
OR The CONTENT_LENGTH is 0
The windir is C:\WINNT
The DeCode length is 0
Now what happened and what parts were involved.
First let's look at the line we typed.
Segment Description Transmission
http
|
Tells the browser what kind of connection to make. This is not info that is
transmitted.
|
://
|
Separator Ends Connection Type Definition
|
localhost
|
Tell the browser the address of the machine to talk to. This is not info that is
transmitted other than to make the connection.
|
:
|
Separator Optional if the port number is given. Default http port number is 80
|
680
|
Tells the browser which port on the server to make the connection request
to.This is not info that is transmitted other than to make the connection.
|
/
|
Separator Terminates Connection Information
|
Cgi/cgimfc1
|
Tells the Bowser the process to run (or page to return) This is part of the
transmitted data
|
?
|
Separator Optional Ends Process definition
|
MyInfo=data1+AndMoreInfo=data2
|
Query Info that the server may use to handle the request. This is part of the
transmitted data. This is called the Query
|
What was sent:
GET /cgi/cgimfc1?MyInfo=data1+AndMoreInfo=data2 HTTP/1.1
Accept: image/gif, image/x-xbitmap, image/jpeg,
image/pjpeg, application/vnd.ms-excel, application/msword,
application/vnd.ms-powerpoint, application/pdf, *
Comments on what was sent
The first line is going to start with either GET or POST.
The lines following this are available via the environment calls.
This a major service that is being provide by the Web Server Software.
The last line in what is sent as the basic block of data is "Content Length" (not shown since
no attached data) if an attached set of data is included with the transmission. The length of the
transmitted additional data is the value on this line. This line is
then terminated with two CR/LF pairs. Following this is the attached data.
Since we just sent a request from the browser, this additional data is not there.
It would be created by either scripting or a program generated http call.
What was returned:
HTTP/1.1 200 OK
Date: Thu, 14 Jun 2001 18:40:09 GMT
Server: Apache/1.3.19 (Win32)
Transfer-Encoding: chunked
Content-Type: text/html
1eb
<HTML><BODY>Hello Me<br><br>Your First Line<br><br>
This is more text
There are 2 Arguments here <br><br>
The Query String is MyQuery=QData<br><br>
The Server Soft is Apache/1.3.19 (Win32)<br><br>
The SERVER_PORT is 980<br><br>
The REQUEST_METHOD is POST<br><br>
The REMOTE_USER is <br><br>
The CONTENT_TYPE is <br><br>
The CONTENT_LENGTH is 23<br><br>
OR The CONTENT_LENGTH is 23<br><br>
The windir is C:\WINNT<br><br>
The DeCode length is 21<br><br>
This is Data to Send!<br><br>
</BODY></HTML>
Comments on what was returned.
Note the header data is read by the receiving program and contains the
information on how the request was handled and any error message that may apply.
Following this information is the HTML data that is displayed by the browser.
The Server CGI Program.
Both Apache and IIS support the HTTP server running a command line program
under the server as a CGI app. (For those not on a NT machine Apache can be installed as an
application and run to provide a learning environment.) In doing this the servers
set up environment variables that the CGI program may ask for and the
"standard in" points to the data stream of the attached data sent with the HTTP
call (this is usually used with POST.) This may be done with the GET call also,
but many servers have a fairly small buffer.
The "standard out" points to the data stream of the data to be sent back. So
all a CGI program needs to be is a command line application that reads the
required environment variables and if needed the attached data stream and
performs the operations required. This includes returning data if a faulty call
was made!
Our first CGI program will be best done if we experiment with what data is
available to get an understanding of the environment variables and reading attached data.
We are learning, right.
We start the Visual Studio environment and ask to create a new Win32 Application.
We will call it Cgi1. I will elect to include MFC because I like the CString class.
Care is needed here not use unhandled portions that cause user messages that
would put the server in a wait state if no operators are available! I.E. This is not
good practice for production code!
You have the choice of leaving the root entry point to being the
int _tmain(int argc, TCHAR* argv[], TCHAR*
envp[])
function or just using a
standard main function. I have chosen to use the standard main function on the
belief it would be more portable. Of course the MFC usage would have to be
removed. My entire program (less the helper encoding functions) is:
#include <stdio.h>
#include "stdafx.h"
#include "Cgi1.h"
#ifdef _DEBUG
#define new DEBUG_NEW
#undef THIS_FILE
static char THIS_FILE[] = __FILE__;
#endif
CWinApp theApp;
using namespace std;
int main(int argc, TCHAR* argv[])
{
int nRetCode = 0;
printf("Content-type: text/html\n\n");
FILE *stream;
stream = fopen( "CGITest.txt", "w" );
printf("<HTML><BODY>");
printf("Hello Me<br><br>");
printf("Your First Line<br><br>\n");
cout << "This is more text<br><br>" << endl;
CString Line;
Line.Format("There are %d Arguments here <br><br>\n",argc);
printf((LPCSTR)Line);
fprintf( stream, "%s\n",Line);
CString path;
path = DeCode(getenv("QUERY_STRING"));
Line.Format("The Query String is %s<br><br>\n",path);
printf((LPCSTR)Line);
fprintf( stream, "%s\n",Line);
path = getenv("SERVER_SOFTWARE");
Line.Format("The Server Soft is %s<br><br>\n",path);
printf((LPCSTR)Line);
fprintf( stream, "%s\n",Line);
path = getenv("SERVER_PORT");
Line.Format("The SERVER_PORT is %s<br><br>\n",path);
printf((LPCSTR)Line);
fprintf( stream, "%s\n",Line);
path = getenv("REQUEST_METHOD");
Line.Format("The REQUEST_METHOD is %s<br><br>\n",path);
printf((LPCSTR)Line);
fprintf( stream, "%s\n",Line);
path = getenv("REMOTE_USER");
Line.Format("The REMOTE_USER is %s<br><br>\n",path);
printf((LPCSTR)Line);
fprintf( stream, "%s\n",Line);
path = getenv("CONTENT_TYPE");
Line.Format("The CONTENT_TYPE is %s<br><br>\n",path);
printf((LPCSTR)Line);
fprintf( stream, "%s\n",Line);
path = getenv("CONTENT_LENGTH");
Line.Format("The CONTENT_LENGTH is %s<br><br>\n",path);
printf((LPCSTR)Line);
fprintf( stream, "%s\n",Line);
long ldata = atol((LPCSTR)path);
Line.Format("OR The CONTENT_LENGTH is %d<br><br>\n",ldata);
printf((LPCSTR)Line);
fprintf( stream, "%s\n",Line);
path = getenv("windir");
Line.Format("The windir is %s<br><br>\n",path);
printf((LPCSTR)Line);
Line="";
CString xstr;
if(ldata>0)
{
fread(xstr.GetBuffer(ldata),sizeof(char),ldata,stdin);
xstr.ReleaseBuffer();
Line = DeCode(xstr);
}
xstr.Format("The DeCode length is %d<br><br>\n",Line.GetLength());
printf((LPCSTR)xstr);
fprintf( stream, "%s\n",xstr);
Line+="<br><br>\n";
printf((LPCSTR)Line);
fprintf( stream, "%s\n",Line);
printf("</BODY></HTML>");
fclose( stream );
return nRetCode;
}
I have used both the "printf" function and the cout << operator in the above.
Actual text returned by this program is:
HTTP/1.1 200 OK
Date: Thu, 14 Jun 2001 18:40:09 GMT
Server: Apache/1.3.19 (Win32)
Transfer-Encoding: chunked
Content-Type: text/html
1eb
<HTML><BODY>Hello Me<br><br>Your First Line<br><br>
This is more text<br><br>
There are 2 Arguments here <br><br>
The Query String is MyQuery=QData<br><br>
The Server Soft is Apache/1.3.19 (Win32)<br><br>
The SERVER_PORT is 980<br><br>
The REQUEST_METHOD is POST<br><br>
The REMOTE_USER is <br><br>
The CONTENT_TYPE is <br><br>
The CONTENT_LENGTH is 23<br><br>
OR The CONTENT_LENGTH is 23<br><br>
The windir is C:\WINNT<br><br>
The DeCode length is 21<br><br>
This is Data to Send!<br><br>
</BODY></HTML>
The header is supplied by the server. It then adds double cr/lf followed by
length of return buffer on next line. Which is then followed by the text your
program wrote out. It is required that your program output the content type for
the server to use as the first output from the program. Following this you need
to bound the remaining text with in the <HTML> and <BODY> tags if you wish to
use a browser to display the results for testing. I have found this useful.
When you are happy with the data you can then uncomment these lines and proceed.
Critical items to note here are the calls to get environment variables and the names of variables you may need.
Is the call being a GET or POST important?
Logging the Port important? What was the "Query String"? How long (if any) is
the attached data. Then go read the data and handle it.
The Client Program (the download)
To get started we just need some basic functionality that all other usage can
be implemented on. We also must be able to view our data to understand what is
really being sent. The web browser is a starting point but we also need to
understand how much processing the browser is doing to create the call to the
server. Several items in the standard URL are never sent but are used to create
the connection to the server. To help clarify this I created a C function which
accepts the components needed and then does the processing of the request. This
function can then be used throughout any application we choose.
To show this usage I will just create a simple Dialog Application with edit
fields for each meaningful item.
The server address and port are needed to make the connection.
That data is never actually sent to the server. Once the connection is made you
must package the data to be sent. The actual data sent is as follows:
POST /cgi-bin/CgiMfc1?MyQuery%3dQData HTTP/1.1
Accept: text/*
User-Agent: HttpCall
Accept-Language: en-us
Host: localhost:680
Content-Length: 98
000629EF2656+vs+%0d%0a143%2e114%2e37%2e22+vs+%0d%0aDEKGKIEG+vs+
%0d%0a000629ef2656+vs+%0d%0a2051586
The Code to make the call in the demo is:
void CHttpAccessDemoDlg::OnSubmit()
{
CWnd* pWndRet = GetDlgItem(IDC_ReturnData);
UpdateData(TRUE);
m_ReturnedData="";
CMemFile MemFile;
int RetVal = HttpCall((LPCSTR)m_ServerAddress,m_ServerPort,
(LPCSTR)m_ServerProcess,(LPCSTR)m_QueryString,
(LPCSTR)m_DataBlock,&MemFile,pWndRet);
int DataLen = MemFile.GetLength();
MemFile.SeekToBegin();
MemFile.Read(m_ReturnedData.GetBuffer(DataLen),DataLen);
m_ReturnedData.ReleaseBuffer();
UpdateData(FALSE);
}
Which calls the general a purpose function "HttpCall" which for this example
uses the MFC CHttpSession and related classes.
int HttpCall(const char* ServerAddress,
int ServerPort,
const char* ServerProcess,
const char* QueryString,
const char* DataBlock,
CFile* DataFile,
CWnd* pStatusWnd)
{
CString strUser;
strUser = GetTheUserName();
long lread,max;
DWORD ServiceType = INTERNET_SERVICE_HTTP;
CString UserName="anonymous";
CString UserPassWord="";
CString lf;
lf.Format("%c%c",char(13),char(10));
CString EnCodeData = EnCodeStr(DataBlock);
DWORD AccessType = PRE_CONFIG_INTERNET_ACCESS;
CString UserDef = "HttpCall";
CHttpSession* pHttpSession = new CHttpSession(UserDef,1,AccessType);
pHttpSession->SetStatusWnd(pStatusWnd);
CHttpConnection* pHttpConnection = NULL;
pHttpConnection = pHttpSession->
GetHttpConnection(ServerAddress,ServerPort,
UserName,UserPassWord);
CHttpFile* pHttpFile = NULL;
DWORD HttpRequestFlags;
CString Process,Query;
Process = ServerProcess;
Query = QueryString;
if(Query.GetLength()>0) Process += '?'+EnCodeStr(Query);
TRY
{
HttpRequestFlags = INTERNET_FLAG_EXISTING_CONNECT |
INTERNET_FLAG_RELOAD | INTERNET_FLAG_DONT_CACHE;
DWORD TotalLen;
TotalLen = EnCodeData.GetLength();
if(TotalLen>0)
{
pHttpFile = pHttpConnection->OpenRequest(CHttpConnection::HTTP_VERB_POST,
Process, NULL, 1, NULL, (LPCTSTR)"1.0", HttpRequestFlags);
}
else
{
pHttpFile = pHttpConnection->OpenRequest(CHttpConnection::HTTP_VERB_GET,
Process, NULL, 1, NULL, (LPCTSTR)"1.0", HttpRequestFlags);
}
CString strHeaders = "Accept: text/*\r\n";
strHeaders += "User-Agent: HttpCall\r\n";
strHeaders += "Accept-Language: en-us\r\n";
pHttpFile->AddRequestHeaders((LPCSTR)strHeaders);
if(TotalLen>0)
{
pHttpFile->SendRequestEx(TotalLen,HSR_INITIATE,1);
pHttpFile->WriteString((LPCTSTR)EnCodeData);
pHttpFile->EndRequest();
}
else
{
pHttpFile->SendRequest();
}
max = pHttpFile->GetLength();
}
CATCH_ALL(e)
{
AfxMessageBox("Connection to Server Failed");
}
END_CATCH_ALL
DWORD dwRet;
pHttpFile->QueryInfoStatusCode(dwRet);
DataFile->SeekToBegin();
CString strRetBufLen;
pHttpFile->QueryInfo(HTTP_QUERY_CONTENT_LENGTH, strRetBufLen);
max = atol((LPCSTR)strRetBufLen);
if(max<=0)
{
max = pHttpFile->GetLength();
}
lread=1000;
char c[1000];
while(lread>0)
{
lread = pHttpFile->Read(c,1000);
if(lread>0) DataFile->Write(c,lread);
}
pHttpFile->Close();
pHttpConnection->Close();
pHttpSession->Close();
delete pHttpFile;
delete pHttpConnection;
delete pHttpSession;
return dwRet;
}
General purpose helper functions:
CString EnCodeStr(CString ToCode)
{
CString RetStr,AddStr;
int i,max;
unsigned short asc;
unsigned char c;
max = (unsigned int)ToCode.GetLength();
for(i=0;i<max;i++)
{
c = ToCode[i];
asc = c;
if(asc>47 && asc<58)
{
RetStr+=c;
}
else if(asc>64 && asc<91)
{
RetStr+=c;
}
else if(asc>96 && asc<123)
{
RetStr+=c;
}
else if(asc==32)
{
RetStr+="+";
}
else
{
AddStr.Format("%%%2x",asc);
int iv = (int)AddStr.GetAt(1);
if((int)AddStr.GetAt(1)==32)
{
AddStr.SetAt(1,'0');
}
RetStr+=AddStr;
}
}
return RetStr;
}
CString DeCodeStr(CString ToCode)
{
CString RetStr,AddStr;
int i,max;
unsigned short asc;
unsigned char c;
max = (unsigned int)ToCode.GetLength();
for(i=0;i<max;)
{
c = ToCode[i];
asc = c;
if(asc==37)
{
AddStr=ToCode.Mid(i+1,2);
i+=3;
sscanf((LPCTSTR)AddStr,"%2x",&asc);
RetStr+=(char)asc;
}
else if(asc==43)
{
RetStr += ' ';
i++;
}
else
{
RetStr += c;
i++;
}
}
return RetStr;
}
Began programming in 1968 on a Wang 720. Move to Fortran and began developing FEM (finite element model) applications on an IBM 360 in 1973. Developed custom FEM editors for most of my career until 1995.
General 
News 
Suggestion 
Question 
Bug 
Answer 
Joke 
Praise 
Rant 
Admin 
Submit an article or tip
If possible, could you please share some of your code with me? Or could you give me some hints?