Handling non-English characters UTF8 with MS VC++ 2005

Introduction

Many times, as a beginner in programming with MS Visual Studio and as a Spanish speaker, I have had problems dealing with non-English characters. This simple and short article describes how to read/write text files using the UTF8 characters map and some problems I had with Visual Studio 2005.

Background

During my university life, all the programming that I learned was orientated toward create background code, not end user programs. So I was familiar with common problems associated with the GCC compiler, and everything worked fine. That was until I had to make a end user app. This is a not very big wall, but a one for many. I want to share how I sorted it.

Using the Code

I first tried to use the basic fstream class, but in VS, the kind of char that is used is the 7-bit char, and when trying to use it with wchar_t, no overloading function errors were shown. So, since that, I decided that I had no more choice than using the famous CLR, not knowing that this was just the beginning of my problems.

I started with the FileStream class, with no great results. The string that is used is a 7-bit (again). Then, finally, I found the StreamRead and StreamWrite classes. Shown here is some simple code to read and write a file using this character map:

#include "stdafx.h"
#using <mscorlib.dll>

using namespace System;
using namespace System::IO;
using namespace System::Text;

int main(array<System::String ^> ^args)
{
    String^ filein;
    String^ fileout;
    Console::WriteLine(L"Enter file name: ");
    filein = Console::ReadLine();
    fileout = "out"+filein;

    StreamReader ^srp = gcnew StreamReader(filein, 
                        System::Text::Encoding::GetEncoding(1252));
    StreamWriter ^swp = gcnew StreamWriter(fileout, false, 
                        System::Text::Encoding::GetEncoding(1252));
    
    swp->WriteLine(L"Some simple header text with á é ä Ñe");
        
    String ^cont ;
    while (cont = srp->ReadLine())
    {
        swp->WriteLine(cont); 

       }
    swp->Close();
    srp->Close();
    delete(IDisposable^) swp;
    delete(IDisposable^) srp;

    return 0;
}

Points of Interest

An interesting point is that the text files generated in this program were well read on the VS file editor. I checked this with the context editor. I think may be this is because of the default language setting of VS.