Click here to Skip to main content
13,588,616 members
Click here to Skip to main content
Add your own
alternative version


12 bookmarked
Posted 11 Jul 2006
Licenced CPOL

Repair Broken Text

, 7 Mar 2007
Rate this:
Please Sign up or sign in to vote.
An article to demonstrate how to repair broken text.


This article demonstrates a way to repair broken text. It puts together a few things tedious to find if you are not aware about them, e.g., how to get characters from the Unicode representation and to use them for replacing strings.


What is meant by broken text? For example, some French text having Cyrillic characters instead of the French specific ones, like "sйchйs, gвteaux" versus the correct ones "séchés, gâteaux". How can you get into such trouble? I got, e.g., because of some legacy data migration.

Using the code

To use the code, just download the sample. The idea is very simple: we need to replace wrong characters with the correct ones; so we define a character map in the XML file MapLiterals.xml, using the Unicode representation of the characters to replace. The code reads the file BrokenText.txt, repairs the text - based on MapLiterals.xml, and saves the repaired text in the file RepairedText.txt.

string text;
FileInfo file = new System.IO.FileInfo("BrokenText.txt");

// Read and repair text
using (TextReader textRead = file.OpenText() )
    text = textRead.ReadToEnd();

    XmlDocument literalsMapXml = new XmlDocument();

    char from, to;
    string buffer;

    XmlNodeList nodes = 
    foreach (XmlNode node in nodes)
        from = (char) ushort.Parse(node.Attributes["from"].Value, 
        to = (char) ushort.Parse(node.Attributes["to"].Value, 

        buffer = text.Replace(from, to);

        text = buffer;

// Write repaired text in a file.
file = new System.IO.FileInfo("RepairedText.txt");
using ( StreamWriter writer = file.CreateText() ) 

Points of interest

By using StringBuilder, memory allocation can be improved.

MapLiterals.xml can contain various character mappings, and it can be defined in several ways like:

  • use hex codes (without '\u') like: from="0439" to="00E9" (the current implementation)
  • use explicit characters like: from= "й" to= "é"; in this case, the code can be simplified like:
// Read and repair text
using (TextReader textRead = file.OpenText())
    string from, to, buffer;
    foreach (XmlNode node in nodes)
        from = node.Attributes["from"].Value;
        to = node.Attributes["to"].Value;

        buffer = text.Replace(from, to);

        text = buffer;

// Write repaired text in a file.

Hope this helps.


This article, along with any associated source code and files, is licensed under The Code Project Open License (CPOL)


About the Author

Software Developer
New Zealand New Zealand

You may also be interested in...

Comments and Discussions

GeneralMy vote of 2 Pin
Jagsir17-Feb-13 12:07
memberJagsir17-Feb-13 12:07 

General General    News News    Suggestion Suggestion    Question Question    Bug Bug    Answer Answer    Joke Joke    Praise Praise    Rant Rant    Admin Admin   

Use Ctrl+Left/Right to switch messages, Ctrl+Up/Down to switch threads, Ctrl+Shift+Left/Right to switch pages.

Permalink | Advertise | Privacy | Cookies | Terms of Use | Mobile
Web03 | 2.8.180615.1 | Last Updated 7 Mar 2007
Article Copyright 2006 by radumi
Everything else Copyright © CodeProject, 1999-2018
Layout: fixed | fluid