Click here to Skip to main content
Click here to Skip to main content

Classes for Writing HTTP Clients in C++

By , 2 Jan 2009
 

Introduction

Today, more and more developers want to write distributed transactional applications for the enterprise and leverage the speed, security, and reliability of server-side technology. Java has become the choice of thousands of developers for writing scalable programs (usually as servlets or JSP) that reside on web servers and execute business methods for clients. (We won't be talking about EJBs here !!)

These clients are usually HTML forms or Java applets that run within a web browser. What if your legacy C++ application wants to go thin, and hand-over all hard jobs like running business logic, accessing database, etc. to a Java servlet? How are you going to handle communication over HTTP, URL encoding variables and all that? The classes we are going to discuss in this article will show you how easy it is to write HTTP clients in C++, even easier than writing a Java Applet for the same purpose!! You take my word for that.

Background

You must be familiar with OOPs, internet protocols, etc. If you have developed a web application using Java, ASP or some other server side technology, then you will find this article easy to follow. To run the demo code, you must have access to a web server that supports Java, I have used Tomcat. If your web server does not support Java, write the server-script in whatever language your web server can interpret and change the server-script's name in the demo code.

Using the Code

This section is actually a walkthrough of the demo project.

First make the following include statements in the CPP file that uses our HTTP classes:

// some standard headers
#include <string>
#include <vector>
#include <iostream>
using namespace std;
// windows headers
#include <windows.h>
#include <wininet.h>
// our header
#include "web.h"
using namespace openutils;

The program must also be linked with wininet.lib. In VC++ you can do this by going to Project-> Settings-> Link-> Object/Library Modules and adding wininet.lib to the end of the default library names.

Next, we declare an object of the WebForm class.

WebForm wf;

You should tell WebForm the name of the web server that it should connect to:

wf.setHost("http://localhost:8080");

You can replace "localhost:8080" with any valid HTTP URL like:

"www.codeproject.com" or "208.45.33.44"

The next parameter that WebForm expects is the name of the script file that the web server should execute:

wf.setScriptFile("/banking/servlet/InterestServlet");

Please check out your web server's documentation for the path of your servlet files. On a Tomcat server, create the folder hierarchy "banking\WEB-INF\classes" in the "webapps" sub-folder and place the IntersetServlet.class file in that.

Now you are ready to add variables to be sent to the servlet for processing. The InterestServlet expects three variables or parameters: name, rate and amt. These variables and their values can be added to the WebForm object by calling the putVariable() function.

wf.putVariable("name","James Dean");
wf.putVariable("rate","14");
wf.putVariable("amt","1200.89");

We can send an HTTP request to the servlet by calling the sendRequest() function.

wf.sendRequest();

The servlet will send back a response that will contain the calculated interest rate in HTML format. This response can be read into a buffer on the client side:

char response[101];            
if(wf.getResponse(response,100)) 
{
    cout << endl << response << endl;
}

We can also use the WebForm class to write simple clients that act like a web-browser. Here is a code snippet that downloads the home page of a well-known web site:

wf.setHost(http://www.microsoft.com);
wf.setScriptFile("/default.html"); 
// or wf.setScriptFile(""); 
wf.sendRequest();
char buff[10 * 1024];
if(wf.getResponse(buff,sizeof(buff))) 
{
    cout << buff << endl;
}
else 
{
    cout << "No response from server" << endl;
}

Notes

web.h defines a utility class for encoding a string in the HTTP encoding format. This class is used internally by WebForm. The usage of this class and the decoder class is demonstrated below:

string str = "AC-0099";
string str_enc = URLEncoder::encode(str); // AC%2d0099
string dec = URLDecoder::decode(str_enc); // AC-0099

Please remember to enclose all web.h function calls in a try-catch block that catch WebFormExceptions.

try {// web.h calls
} catch(WebFormException ex)
{ 
  cout << ex.getMessage(); 
}

Before running the demo, copy InterestServlet.class from the servlet folder to the appropriate script folder of your Java enabled web-server.

History

  • 10/20/2003: Submitted to CodeProject
  • 12/30/2008: Updated source and demo project

License

This article, along with any associated source code and files, is licensed under The BSD License

About the Author

AnOldGreenHorn
India India
Member
No Biography provided

Sign Up to vote   Poor Excellent
Add a reason or comment to your vote: x
Votes of 3 or less require a comment

Comments and Discussions

 
You must Sign In to use this message board.
Search this forum  
    Spacing  Noise  Layout  Per page   
GeneralMy vote of 1memberhighfestiva22 Jan '12 - 9:09 
The source holds low quality, for instance in basic type conversion, is lengthy and non-portable. As an example, take the if statement in line 40. Four comparisons would suffice, instead of using 38... and that's not even the worst part.
QuestionHow can I add my edition of WebForm or send it to the author?memberboriskamenov15 Jul '10 - 22:30 
I have updated the code so it now supports GET and POST and I want to share my changes. But I do not how.
Please help me so I can post the new version.
AnswerRe: How can I add my edition of WebForm or send it to the author?memberVijay Mathew Pandyalakal17 Jul '10 - 6:58 
Please send your modifications to me. I will update the article.
GeneralRe: How can I add my edition of WebForm or send it to the author?memberMember 766385418 Feb '11 - 2:59 
This doesn't seem to have happened Frown | :-(
GeneralTHANKS YOU!!!!member0xFFFFSERG14 Jan '10 - 16:16 
You're my savior, thanks a lot!!!
I love you )))))
QuestionFile uploads?memberArvindRSingh20 Jun '09 - 11:02 
Any way to specify HTTP methods? Or any other way to upload a file through an application to server side script/servlet accepting a file?
GeneralWeb Server ConfigmemberMember 213022722 Jan '09 - 4:15 
Thanks for the example code.
 
How do you get the web server (in my case Tomcat running on windows) to execute the Java code on the server. I am sure this is a configuration question about Tomcat and Java and maybe this is outside the bounds of this topic.
 
I have managed to create a web server (using Tomcat) and installed Java, however the web server will not run the Java code - it just returns the contents of the files when I fully name the files (e.g. wf.setScriptFile("/InterestServlet.class") but fails to execute them when I use wf.setScriptFile("/InterestServlet") and returns a file not found message;
 
I suspect I am missing the configuration that tells Tomcat to use Java and execute the file.
 
If anyone could point me to a URL with some examples then that would really help me.
 
Regards
Alan
GeneralAsyncmemberjsh_ec3 Jan '09 - 5:34 
Thanks for your article. In the real world, we would normally be using async functionality (for timeout and efficiency).
 
There are a few errors in the code:
* calling sendRequest twice will recreate the session and request handle
* hex doesn't have the letter 'g'
* use >= 'a' <= 'z' >= '0' <= '9' in isOrdinaryChar
GeneralProblem with opening "big" sitesmemberMilos Milovanovic30 Dec '08 - 7:36 
Hi Vijay,
 
Your article is great! However I have problem with “big” sites. For example I have problem with following code:
 
// some standard headers
#include
#include
#include
using namespace std;
// windows headers
#include
#include
// our header
#include "web.h"
using namespace openutils;
int main()
{
WebForm wf;
wf.setHost("http://www.belex.rs");
wf.setScriptFile("/srp/akcije.php");
wf.putVariable("t","2");
wf.putVariable("s","AGBN");
wf.putVariable("i","770");
wf.sendRequest();
char buff[400 * 1024];
if(wf.getResponse(buff,sizeof(buff)-1))
{
cout << buff << endl;
cout << "Response length: " << strlen(buff) << endl;
}
else
{
cout << "No response from server" << endl;
}
return 0;
}
 
This code will return around 26k characters, but webpage has around 41k characters. I also used Fiddler, Web Debugging Proxy, (www.fiddlertool.com) , to see if HTTP request and response are correct and they are. I see in Fiddler correct web page, but in my program I don’t receive all character. Do you have any idea what is the problem?

GeneralRe: Problem with opening "big" sitesmemberVijay Mathew Pandyalakal30 Dec '08 - 17:15 
Hi Milos,
 
I found a bug in the getResponse() function. Thanks for bringing this up. You need to modify WebForm::getResponse() as follows:
 
bool WebForm::getResponse(char* buff,int sz) {
	if(m_hRequest == NULL) {
		throw WebFormException("No request made to server !");
		return false;
	}	
	int totalBytesRead = 0;
	int origSize = sz;
	std::string buffer;
	while (totalBytesRead < origSize) {
		if (InternetReadFile(m_hRequest,buff,sz,&m_lBytesRead)) {
			if (m_lBytesRead > sz) {
				throw WebFormException("Buffer overflow !");				
			}
			if (m_lBytesRead == 0)
				break;
			buff[m_lBytesRead] = 0;
			buffer += buff;
			strcpy(buff, "");
			totalBytesRead += m_lBytesRead;
			sz -= m_lBytesRead;
			if (sz <= 0)
				break;
		}
	}
	strcpy(buff, buffer.c_str());
	return true;	
}
 
Let me know the result. I will be posting the modified code soon.
 
Thanks,
 
-- Vijay
GeneralRe: Problem with opening "big" sitesmemberMilos Milovanovic1 Jan '09 - 2:37 
Thank you Vijay,
 
Now this works! Great!
 
I have one more question. Is it possible to know the size of the response, before getResponse method is executed? In order to allocate buffer with sufficient size I need that information (I tried method getBytesRead but it doesn't work properly). Maybe you should add one more method for that information. Or if that is not possible some other mechanism form chunking of response should be introduced.
 
Thank you in advance!
 
Milos
 
P.S. Happy New Year! Smile | :)
GeneralRe: Problem with opening "big" sitesmemberVijay Mathew Pandyalakal1 Jan '09 - 5:55 
Knowing the size of a HTTP response in advance is not possible. I don't this the protocol provide that facility. What you can do is, write a different version of the getResponse() function, that will return false when no more data can be read and call it in a loop. You can pass an optimum sized buffer to this function (like buff[1024]) and append the data read to a string.
 
Thanks,
 
-- Vijay
GeneralHTTP POSTmemberanthony.kho17 Dec '08 - 22:15 
Hi, is it possible for the code to implement HTTP POST for cgi command?
 
Thank you
GeneralRe: HTTP POSTmemberVijay Mathew Pandyalakal19 Dec '08 - 1:54 
Yes it is possible.
GeneralRe: HTTP POSTmemberSirDiesALot17 Mar '09 - 6:13 
Any idea how you would alter the code to do an HTTP post? I'm kind of lost trying to add that ability.
GeneralRe: HTTP POSTmemberVijay Mathew Pandyalakal17 Mar '09 - 6:45 
Read the spec - http://www.w3.org/Protocols/[^], implement and share !
QuestionHow to handle file not found (HTTP Error 404) ??memberAbinThomas3 Jan '08 - 23:33 
Hello,
Thank you very much for sharing this wonderful peice of code. I am now using it to download certain files. But i need some help.
If the requested file is not in the path, getResponse returns the html content of "HTTP Error 404 - File or directory not found".
How can we check wheher the download was succesfull? Otherwise how to check if the file exists bfor attemting download ?
 
Thanks
abin.
AnswerRe: How to handle file not found (HTTP Error 404) ??memberVijay Mathew Pandyalakal7 Jan '08 - 17:51 
If the requested file is not found, the web server will send you a response like:
 
HTTP/1.0 404 Not Found
 
You need to check for the error code 404. I don't think HTTP supports querying or listing of files.
GeneralThank you for the code + remarkmemberwakeboarderxxx19861 Nov '07 - 9:03 
if(ch == 'a' || ch == 'b' || ch == 'c' || ch == 'd' || ch == 'e'
|| ch == 'f' || ch == 'g' || ch == 'h' || ch == 'i' || ch == 'j'
|| ch == 'k' || ch == 'l' || ch == 'm' || ch == 'n' || ch == 'o'
|| ch == 'p' || ch == 'q' || ch == 'r' || ch == 's' || ch == 't'
|| ch == 'u' || ch == 'v' || ch == 'w' || ch == 'x' || ch == 'y'
|| ch == 'z' || ch == '0' || ch == '1' || ch == '2' || ch == '3'
|| ch == '4' || ch == '5' || ch == '6' || ch == '7' || ch == '8'
|| ch == '9')
 
This could have been written more efficiënt/beautifull using range checking i guess Wink | ;-)
 
if(ch >= 'a' && ch <= 'z' || ch >= '0' && ch <= '9')
 
Nice code and thank you!!
 
Grtz
GeneralAuthentication in HTTP clientsmemberpelis23 May '07 - 3:55 
Hi Mathew,
 
I have been testing your application and it works fine but the server responds with "401 Authorization Required" because I need to login in the system. Is this feature available in your application? Could you lead me how to do it? Because I don't know how to modify the HTTP headers! Actually, i have the code in delphi an i am trying to port it to C++, in delphi the key line is:
 
IdHTTP.Request.ExtraHeaders.Add('Authorization: Basic '+ Base64Encode(Login.Text+':'+Password.Text));

 
Thank you very much in advance!!!!
 
JRF
GeneralRe: Authentication in HTTP clientsmemberVijay Mathew Pandyalakal23 May '07 - 17:38 
As of now, my HTTP library do not directly support user specified headers.
You can easily add this feature by modifying the call to the win32 API
function "InternetOpenUrl" in web.cpp. Additional headers can be given as the third argument of this function. In the current code it is left as NULL.
See the MSDN documentation on InternetOpenUrl() for more information and modify web.cpp to suit your requirement.
 
thanks,
Vijay
General[Linker error] undefined reference to `openutils::WebForm::WebForm()'memberlil_zee27 Feb '07 - 5:05 
hello,
 
when i try to compile your demo or try to write something using your code i get errors like:
 
[Linker error] undefined reference to `openutils::WebForm::WebForm()'
 
I think it has something to do with the wininet.lib which i do not have...
GeneralRe: [Linker error] undefined reference to `openutils::WebForm::WebForm()'memberVijay Mathew Pandyalakal27 Feb '07 - 17:11 
This seems to have nothing to do with wininet.lib
Just make sure that the file 'web.cpp' is getting compiled and linked properley.
GeneralIt does not work got all URLsmemberAndyzyx12 Feb '07 - 4:37 
Hi,
 
It looks cool.
However, when I tried your program for some reason it does not seem to encode well for big sites[ say like http://www.yahoo.com ] and bails out with a "stack around the 'tmp' variable is corrupted" in encode().
 
Anything you would like to suggest to make this work for bigger sites as well.
 
Thanks,
Andy

GeneralRe: It does not work got all URLsmemberVijay Mathew Pandyalakal12 Feb '07 - 18:56 
Hi,
 
Thanks for trying out my code.
I tested it against big sites like yahoo and did not get the error you reported. But from your post I understand that the code will fail under certain cirtcumstances. Could you please post the code that led to the above error?
 
Thanks,
 
Vijay

General General    News News    Suggestion Suggestion    Question Question    Bug Bug    Answer Answer    Joke Joke    Rant Rant    Admin Admin   

Permalink | Advertise | Privacy | Mobile
Web04 | 2.6.130516.1 | Last Updated 3 Jan 2009
Article Copyright 2003 by AnOldGreenHorn
Everything else Copyright © CodeProject, 1999-2013
Terms of Use
Layout: fixed | fluid