Click here to Skip to main content
11,638,173 members (78,785 online)
Click here to Skip to main content

Code to extract plain text from a PDF file

, 21 Jun 2004 514.8K 24.8K 167
Rate this:
Please Sign up or sign in to vote.
Source code that shows how to decompress and extract text from PDF documents.

Introduction

PDF documents are commonly used and their content is usually compressed. This article shows a simple C code that can be used to extract plain text from the PDF file.

Why?

Adobe does allows you to submit PDF files and will extract the text or HTML and mail it back to you. But there are times when you need to extract the text yourself or do it inside an application. You may also want to apply special formatting (e.g., add tabs) so that the text can be easily imported into Excel for example (when your PDF document mostly contains tables that you need to port to Excel, which is how this code got developed).

There are several projects on "The Code Project" that show how to create PDF documents, but none that provide free code that shows how to extract text without using a commercial library. In the reader comments, a need was expressed for code just like what is being supplied here.

There are several libraries out there that read or create PDF file, but you have to register them for commercial use or sign various agreements. The code supplied here is very simple and basic, but it is entirely free. It only use the ZLIB library which is also free.

Basics

You can download documents such as PDFReference15_v5.pdf from here that explains some of the inners of PDF files. In short, each PDF file contains a number of objects. Each object may require one or more filters to decompress it and may also provide a stream of data. Text streams are usually compressed using the FlateDecode filter and may be uncompressed using code from the ZLIB (http://www.zlib.org/) library.

The data for each object can be found between "stream" and "endstream" sections. Once inflated, the data needs to be processed to extract the text. The data usually contains one or more text objects (starting with BT and ending with ET) with formatting instructions inside. You can learn a lot from the structure of PDF file by stepping through this application.

About Code

This single source code file contains very simple, very basic C code. It initially reads in the entire PDF file into one buffer and then repeatedly scans for "stream" and "endstream" sections. It does not check which filter should be applied and always assumes FlateDecode. (If it gets it wrong, usually no output is generated for that section of the file, so it is not a big issue). Once the data stream is inflated (uncompressed), it is processed. During the processing, the code searches for the BT and ET tokens that signify text objects. The contents of each is processed to extract the text and a guess is made as to whether tabs or new line characters are needed.

The code is far from complete or being any sort of general utility class, but it does demonstrate how you can extract the text yourself. It is enough to show you how and get you going.

The code is however fully functional, so when it is applied to a PDF document, it generally does a fair job of extracting the text. It has been tested on several PDF files.

This code is supplied as is, no warranties. Use at your own risk.

Using The Code

The download contains one C file. To use it, create a simple Windows 32 Console project and add the pdf.c file to the project. You also need to go here (bless them!) and download the free "zlib compiled DLL" zip file. Extract zdll.lib to your project directory and add it as a project dependency (link against it). Also put zlib1.dll in your project directory. Also put zconf.h and zlib.h in your project directory and add them to the project.

Now, step through the application and note that the input PDF and output text file names are hardwired at the start of the main method.

Future Enhancements

If there is enough interest, the author may consider uploading a release version with a Windows interface. The code is quite good for extracting data from tables in a form that can be readily imported into Excel, with the column preserved (because of the tabs that get added).

Code Snippets

Stream sections are located using initially:

size_t streamstart = FindStringInBuffer (buffer, "stream", filelen);
size_t streamend = FindStringInBuffer (buffer, "endstream", filelen);

And then once the data portion is identified, it is inflated as follows:

z_stream zstrm; ZeroMemory(&zstrm, sizeof(zstrm));
zstrm.avail_in = streamend - streamstart + 1;
zstrm.avail_out = outsize;
zstrm.next_in = (Bytef*)(buffer + streamstart);
zstrm.next_out = (Bytef*)output;
int rsti = inflateInit(&zstrm);
if (rsti == Z_OK)
{
  int rst2 = inflate (&zstrm, Z_FINISH);
  if (rst2 >= 0)
  {
    //Ok, got something, extract the text:
    size_t totout = zstrm.total_out;
    ProcessOutput(fileo, output, totout);
  }
}

The main work gets done in the ProcessOutput method which processes the uncompressed stream to extract text portion of any text object. It looks as follows:

void ProcessOutput(FILE* file, char* output, size_t len)
{
  //Are we currently inside a text object?
  bool intextobject = false;
  //Is the next character literal 
  //(e.g. \\ to get a \ character or \( to get ( ):
  bool nextliteral = false;

  //() Bracket nesting level. Text appears inside ()
  int rbdepth = 0;

  //Keep previous chars to extract numbers etc.:
  char oc[oldchar];
  int j=0;
  for (j=0; j<oldchar; j++) oc[j]=' ';

  for (size_t i=0; i<len; i++)
  {
    char c = output[i];
    if (intextobject)
    {
      if (rbdepth==0 && seen2("TD", oc))
      {
        //Positioning.
        //See if a new line has to start or just a tab:
        float num = ExtractNumber(oc,oldchar-5);
        if (num>1.0)
        {
          fputc(0x0d, file);
          fputc(0x0a, file);
        }
        if (num<1.0)
        {
          fputc('\t', file);
        }
      }
      if (rbdepth==0 && seen2("ET", oc))
      {
        //End of a text object, also go to a new line.
        intextobject = false;
        fputc(0x0d, file);
        fputc(0x0a, file);
      }
      else if (c=='(' && rbdepth==0 && !nextliteral) 
      {
        //Start outputting text!
        rbdepth=1;
        //See if a space or tab (>1000) is called for by looking
        //at the number in front of (
        int num = ExtractNumber(oc,oldchar-1);
        if (num>0)
        {
          if (num>1000.0)
          {
            fputc('\t', file);
          }
          else if (num>100.0)
          {
            fputc(' ', file);
          }
        }
      }
      else if (c==')' && rbdepth==1 && !nextliteral) 
      {
        //Stop outputting text
        rbdepth=0;
      }
      else if (rbdepth==1) 
      {
        //Just a normal text character:
        if (c=='\\' && !nextliteral)
        {
          //Only print out next character 
          //no matter what. Do not interpret.
          nextliteral = true;
        }
        else
        {
          nextliteral = false;
          if ( ((c>=' ') && (c<='~')) || ((c>=128) && (c<255)) )
          {
            fputc(c, file);
          }
        }
      }
    }
    //Store the recent characters for 
    //when we have to go back for a number:
    for (j=0; j<oldchar-1; j++) oc[j]=oc[j+1];
      oc[oldchar-1]=c;
    if (!intextobject)
    {
      if (seen2("BT", oc))
      {
        //Start of a text object:
        intextobject = true;
      }
    }
  }
}

License

This article has no explicit license attached to it but may contain usage terms in the article text or the download files themselves. If in doubt please contact the author via the discussion board below.

A list of licenses authors might use can be found here

Share

About the Author

NeWi
Web Developer
Canada Canada
No Biography provided

You may also be interested in...

Comments and Discussions

 
QuestionIt doesn't work (unless my version is wrong, (latest download)) Pin
DaveLeikis13-May-15 7:04
memberDaveLeikis13-May-15 7:04 
Question.tar compressed instead Pin
DaveLeikis20-Mar-15 6:01
memberDaveLeikis20-Mar-15 6:01 
BugProcedure seen2() need to take into account TAB separator Pin
Giovefi14-Jul-14 14:41
memberGiovefi14-Jul-14 14:41 
SuggestionIdentity-H without Table Pin
xOLIVERx21-Nov-13 22:27
memberxOLIVERx21-Nov-13 22:27 
QuestionLinking error please help .... Pin
arman78721-Nov-13 2:31
memberarman78721-Nov-13 2:31 
AnswerRe: Linking error please help .... Pin
arman7875-Dec-13 2:24
memberarman7875-Dec-13 2:24 
Questionthanks for sharing Pin
jacaboo15-Sep-13 16:21
memberjacaboo15-Sep-13 16:21 
QuestionThank you Pin
Member 99624943-Apr-13 9:12
memberMember 99624943-Apr-13 9:12 
Question[My vote of 1] ! Pin
i30mb16-Feb-13 21:01
memberi30mb16-Feb-13 21:01 
QuestionLeila Pin
Rashidy_30-Dec-12 11:02
memberRashidy_30-Dec-12 11:02 
QuestionVirus in the setup.exe Pin
Andrew Guest15-Nov-12 2:39
memberAndrew Guest15-Nov-12 2:39 
BugA tiny and a less tiny bug in the Source code Pin
Gerald Schade28-Oct-12 22:08
memberGerald Schade28-Oct-12 22:08 
QuestionDOC to PDF conversion Pin
Member 93319263-Sep-12 21:39
memberMember 93319263-Sep-12 21:39 
hi,
I want to convert a word file to pdf format.
i need complete code for this.
will u please help me.
i am using VS2010.
thanks in advance..
QuestionIncorrect header check Pin
fjsanzano2-May-12 5:12
memberfjsanzano2-May-12 5:12 
AnswerRe: Incorrect header check Pin
Rolcsar25-Sep-12 6:22
memberRolcsar25-Sep-12 6:22 
GeneralMy vote of 5 Pin
manoj kumar choubey26-Feb-12 19:59
membermanoj kumar choubey26-Feb-12 19:59 
QuestionSearch a String from PDF file Pin
Spoorti Hallur22-Feb-12 23:07
memberSpoorti Hallur22-Feb-12 23:07 
GeneralMy vote of 5 Pin
markjuggles29-Jan-12 11:45
membermarkjuggles29-Jan-12 11:45 
GeneralBrillaint idea Pin
Member 305875510-Aug-11 5:38
memberMember 305875510-Aug-11 5:38 
Generallinking error using C++ Pin
tsubaryanidhs12-Dec-10 22:08
membertsubaryanidhs12-Dec-10 22:08 
GeneralRe: linking error using C++ Pin
cute_friend70779-Jun-11 20:18
membercute_friend70779-Jun-11 20:18 
GeneralWindows Forms Application Pin
adsjoom1-Dec-10 6:27
memberadsjoom1-Dec-10 6:27 
GeneralRe: Windows Forms Application Pin
tsubaryanidhs1-Jan-11 8:18
membertsubaryanidhs1-Jan-11 8:18 
GeneralMy vote of 5 Pin
Geek Master6-Oct-10 7:18
memberGeek Master6-Oct-10 7:18 
GeneralMy vote of 5 Pin
Allen Friend2-Oct-10 12:28
memberAllen Friend2-Oct-10 12:28 
GeneralNot able to find zdll.lib Pin
Member 446280510-Jun-10 20:29
memberMember 446280510-Jun-10 20:29 
GeneralRe: Not able to find zdll.lib Pin
tiutababo15-Jun-10 21:27
membertiutababo15-Jun-10 21:27 
GeneralRe: Not able to find zdll.lib Pin
Tilman Hausherr5-Aug-10 8:57
memberTilman Hausherr5-Aug-10 8:57 
Generaldoes not support embedded fonts Pin
lipoor19-May-10 21:01
memberlipoor19-May-10 21:01 
QuestionCompatibility with Mac Pin
graccus29-Mar-10 9:37
membergraccus29-Mar-10 9:37 
GeneralTHANK YOU!!! Pin
marceloflu18-Feb-10 8:17
membermarceloflu18-Feb-10 8:17 
GeneralGetting Blank Command prompt Pin
gokul15074-Jan-10 23:33
membergokul15074-Jan-10 23:33 
Generaldiacritics Pin
tontoncaidd21-Sep-09 4:38
membertontoncaidd21-Sep-09 4:38 
GeneralA pascal translation of this pdf text extractor [modified] Pin
Domingo Alvarez29-Jul-09 13:25
memberDomingo Alvarez29-Jul-09 13:25 
GeneralA less complex pascal translation of this pdf text extractor that add spaces betwen lines Pin
Domingo Alvarez30-Jul-09 23:32
memberDomingo Alvarez30-Jul-09 23:32 
GeneralUnable to delete the buffer Pin
REDSERPENT712-Mar-09 1:04
memberREDSERPENT712-Mar-09 1:04 
Generalthanx Pin
mrares26-Feb-09 13:02
membermrares26-Feb-09 13:02 
General.Net version Pin
nstuart23-Feb-09 10:37
membernstuart23-Feb-09 10:37 
AnswerRe: .Net version Pin
Vasiliy Zverev29-Sep-10 6:37
memberVasiliy Zverev29-Sep-10 6:37 
GeneralLinking error : on deflateInit(....) call Pin
mayur8u11-Feb-09 16:58
membermayur8u11-Feb-09 16:58 
GeneralRe: Linking error : on deflateInit(....) call Pin
alirezazarei8-Sep-09 13:39
memberalirezazarei8-Sep-09 13:39 
QuestionProblem with implementation ( help ) Pin
ankit0921-Jan-09 2:43
memberankit0921-Jan-09 2:43 
QuestionHow to extract it page by page, or rather how to detect a new page? Pin
Alexander Schmidt6-Dec-08 7:15
memberAlexander Schmidt6-Dec-08 7:15 
AnswerRe: How to extract it page by page, or rather how to detect a new page? Pin
themanof8310-Dec-08 5:41
memberthemanof8310-Dec-08 5:41 
QuestionReading PDF files with tables Pin
Leon Stenneth24-Nov-08 18:32
memberLeon Stenneth24-Nov-08 18:32 
GeneralBug in code... Pin
nortonio29-May-08 4:27
membernortonio29-May-08 4:27 
GeneralRe: Bug in code... Pin
Tilman Hausherr5-Aug-10 8:42
memberTilman Hausherr5-Aug-10 8:42 
Generalthank you (RESPECT PACA PACA) Pin
leonel2014-May-08 7:16
memberleonel2014-May-08 7:16 
GeneralRussian text [modified] Pin
Smolensk13-May-08 20:27
memberSmolensk13-May-08 20:27 
GeneralLink Error Pin
avinash_ss5-Mar-08 19:11
memberavinash_ss5-Mar-08 19:11 

General General    News News    Suggestion Suggestion    Question Question    Bug Bug    Answer Answer    Joke Joke    Rant Rant    Admin Admin   

Use Ctrl+Left/Right to switch messages, Ctrl+Up/Down to switch threads, Ctrl+Shift+Left/Right to switch pages.

| Advertise | Privacy | Terms of Use | Mobile
Web03 | 2.8.150728.1 | Last Updated 22 Jun 2004
Article Copyright 2004 by NeWi
Everything else Copyright © CodeProject, 1999-2015
Layout: fixed | fluid