There is no doubt that imaging technologies such as forms recognition and OCR have revolutionized the way we do business and have freed up countless man-hours from the tedious and error-prone task of data entry. However, these technologies still rely on the assumption that the data will be located in the same place on every document. Unstructured forms and scanned documents such as invoices, bills and tabs are notoriously difficult, if not impossible, to process on a large scale within an automated system.
This is no longer the case after the release of LEADTOOLS Version 19 and its advanced Invoice Recognition and Processing SDK technology. Complex, unstructured and semi-structured documents with tabular data can be processed even when each row and cell have different heights, or when the table spans across multiple pages. In this white paper we will show how to set up a master form template then extract the data from a filled invoice.
Setting up the Master Form Template
The first requirement is to create a template which defines where the various fields are located and what kind of data to expect. This should be a blank or redacted version of the form to recognize. Some fields may have static locations such as the customer number, invoice number, addresses, etc., but tables found on invoices and other unstructured form types can wreak havoc on recognition accuracy since the master template and filled invoice differ in size due to the table area.
Figure 1: Master Form Template compared to a Filled Invoice
LEADTOOLS overcomes this major obstacle with the Table field type. To implement a table field, simply define the column headers and the table data will be recognized regardless of the number of rows, height of each cell, or the number of pages. Traditional fixed location fields can also coexist underneath the table (e.g. Invoice Total) and will be discovered based on its location in proportion to the end of the table.
Figure 2: LEADTOOLS Master Form Editor Demo editing a master form with a table field in addition to traditional static fields
Recognizing and Processing a Filled Invoice
After setting up the master form templates, you are ready to start recognizing (i.e. classifying) and processing filled out documents. LEADTOOLS' advanced forms recognition algorithm will first determine what kind of form the filled document matches. There is no limit to the number of master form templates within a repository so it is possible to create a single, automated solution for all of the forms and invoices within an enterprise. The
AutoFormsEngine does all of the work of aligning, orienting, classifying and processing the form regardless of its resolution. Best of all, setting up the
AutoFormsEngine is quick and easy, requiring little more than telling it where to find the master form templates and passing it the filled invoice image.
DiskMasterFormsRepository workingRepository = new DiskMasterFormsRepository(rasterCodecs,
AutoFormsRecognitionManager managers = AutoFormsRecognitionManager.Ocr;
AutoFormsEngine autoEngine = new AutoFormsEngine(workingRepository, ocrEngines, barcodeEngine,
managers, 30, 80, _menuItemRecognizeFirstPageOnly.Checked);
autoEngine.UseThreadPool = true;
FilledForm form = new FilledForm();
form.FileName = Path.GetFileName(imagePath);
form.Name = Path.GetFileNameWithoutExtension(imagePath);
form.Image = rasterCodecs.Load(imagePath, 0, CodecsLoadByteOrder.Bgr, 1, 1);
AutoFormsRunResult result = autoEngine.Run(form.Image, MyPageRequestCallback, form, null);
if (result != null && !canceled)
MasterForm master = new MasterForm();
master.Properties = result.RecognitionResult.Properties;
form.Master = master;
form.Result = result.RecognitionResult.Result;
form.Alignment = result.RecognitionResult.Alignments;
After a form is successfully classified, we can parse the results and extract data from the fields defined by the template.
_tableResults = new DataGridView();
TableFormField tableField = form.ProcessingPages[pageIndex][fieldIndex] as TableFormField;
if (tableField.Result.Status == FormFieldStatus.Success)
foreach (TableColumn column in tableField.Columns)
TableFormFieldResult results = tableField.Result as TableFormFieldResult;
for (int i = 0; i < results.Rows.Count; i++)
TableFormRow row = results.Rows[i];
for (int j = 0; j < row.Fields.Count; j++)
OcrFormField ocrField = row.Fields[j];
TextFormFieldResult txtResults = ocrField.Result as TextFormFieldResult;
_tableResults.Rows[i].Cells[j].Value = txtResults.Text;
Each field contains a plethora of information including the recognized data, confidence information, font characteristics, bounding rectangle and more. As you can see in Figure 3, the entire table is detected as a field, and is further broken down into each cell. LEADTOOLS' flexible and highly accurate processing routine is able to detect and account for variances in cell height, properly identifying the bounding rectangle of each cell and using optical character recognition to extract the data and detailed character information.
Figure 3: Displaying the Forms Recognition and Processing results
Want to see more? Click here for a video demonstration of the LEADTOOLS Invoice Recognition and Processing SDK.
Recognizing and processing data from unstructured forms such as invoices and bills is just one of many real-world solutions you can tackle with LEADTOOLS. Its state-of-the-art Invoice Recognition and Processing SDK makes it possible to create automated accounting, billing and invoicing solutions. Furthermore, LEADTOOLS provides a bountiful collection of document cleanup functions including deskew, line remove, despeckle, hole punch remove, etc., to filter out noise prior to recognition. LEADTOOLS offers an incredible value with its comprehensive family of toolkits for raster, document, medical and multimedia imaging.
Download the Full Unstructured Invoice Recognition and Processing Examples
You can download the fully functional demo which includes the features discussed above. To run this example you will need the following:
- LEADTOOLS free 60 day evaluation
- Visual Studio 2008 or later
- Browse to the LEADTOOLS Examples folder (e.g. C:\LEADTOOLS 19\Examples\) where you can find example projects for this and many more technologies in LEADTOOLS
Need help getting this sample up and going? Contact our support team for free technical support! For pricing or licensing questions, you can contact our sales team (firstname.lastname@example.org) or call us at 704-332-5532.