Rendering PDF Documents with Mupdf and P/Invoke in C#

Introduction

I've been looking for a PDF rendering engine on .NET Framework for quite a few years. However, there is NO free, native .NET based one available on this planet yet. At the end, I was attracted by a GNU licensed, fast and slim PDF rendering engine--Mupdf, and wrote my own C# wrapper for it.

This article will introduce the first steps on writing such a P/Invoke based C# wrapper for the Mupdf engine and render PDF pages into picture files.

Background and relative projects

During the search for PDF render engines, I found two projects providing C# interfaces nicely. The libraries provided by these two projects can be used to render PDF pages with C#.

The first project was initially posted here on CodeProject.com, titled View PDF files in C# using the Xpdf and muPDF and finally settled down on GoogleCode, named pdfviewer-win32. It used C++/CLI to bridge C# and the Mupdf. The positive side of the C++/CLI approach is that it allows developers to access every most internal parts of the Mupdf library. The down side is that developers have to write many C++/CLI wrapper classes. Writing and debugging such kind of classes isn't a piece of cake, and many C# developers even don't know about C++.

The second project was hosted on GoogleCode, named mupdf-converter. It has 2 layers: the first one introduces some C++ classes to encapsulate the library; the second one provides some C functions as static methods exposed by those classes. Finally, those C functions are exported in a DLL for P/Invoke. In short, the flow of the wrapper is MuPDF -> C++ Wrapper -> C (exported as DLL functions) -> .NET P/Invoke.

Since there're existing projects such as pdfviewer-win32 or mupdf-converter, why not just use them? It was fine unless you want to keep up with the most recent development or use all functionalities provided by the Mupdf library, when you have to eventually dig into the Mupdf yourself. The pdfviewer-win32 project provides a lot of functionalities, but it had not been updated for more than one year when I wrote this article. The mupdf-converter library provides merely very few functionalities--You could not do other things with it except rendering PDF pages--and it works only on .NET 4.0 or above.

I studied the source code of mupdf-converter and found that it was quite simple and its two-layer-wrapper implementation was quite bloated. Actually it is unnecessary to use a C++ class to wrap functions from Mupdf. As Mupdf is written in C, it is possible to make a DLL out of it and call its export functions via P/Invoke directly. That is, the calling path can be shorten to MuPDF -> .NET P/Invoke. Inspired by this library, I began to make my own and hence wrote this article.

Warning: The following part of the article assumes that you know how to write P/Invoke functions in C#. If you don't know a bit about it, you can learn it from MSDN. If you don't want to take that trouble. Please use the existing library provided by pdfviewer-win32 or mupdf-converter.

The 'cooking' procedures

The procedure of making our P/Invoke-only MupdfSharp library includes the following steps.

Obtaining the Mupdf DLL library.
Learning the essential concepts and export functions in Mupdf.
Writing the P/Invoke functions.

Obtaining the Mupdf DLL

If you have once downloaded and compiled the source code of Mupdf, you would probably find that the compiled output are several EXE files. There is no DLL library at all. So if you are not familiar with MAKE files, it can take you quite some hours to learn and modify the MAKE file in order to get the DLL library.

Fortunately there are always nice guys in this world. The developers of SumatraPDF, a slim PDF viewer program utilizing the power of Mupdf, are our savers. They have released their code files on GoogleCode and the compile result of their project does lay down a DLL library for you to reuse. So the steps can be quite simple.

Goto the project host of SumatraPDF: http://code.google.com/p/sumatrapdf/
Download the source code package (or use an SVN tool to synchronize with their latest work).
Open Visual C++ (you can use the free Express version here) and load the project.
Select "Release" as the build configuration and compile the project.
Find out the "libmupdf.dll" file out of the release folder.
You have the Mupdf DLL now.

The developers of SumatraPDF are two of the most diligent programmers. They keep quit a closed track with the latest development of Mupdf and update their code frequently. Therefore, you can quite well trust them and use their library instead of trying to compile your own Mupdf DLL out of the official code.

Learning the essential concepts and functions

Once you have the libmupdf.dll in your hands. It is time for you to study the functions provided by the Mupdf library.

The library is written in C. The most common functions are placed in one header file--"fitz.h". Although Mupdf supports several file formats besides PDF, the fitz.h file is sufficient for us to get started. If you don't have the source code of Mupdf, you can access the content of fitz.h from the documentation section of the official website.

Five key structures in fitz.h are used in our code, listed as below.

fz_context structure: Used to hold information when working with PDF files (or other supported document formats). You have to prepare the context before working.
fz_document structure: Used to hold the opened PDF document.
fz_page structure: Used to work with PDF pages. Once you opened a document, you can load its pages and work with them.
fz_pixmap structure: Represents the rendered visual results of a page. If you want to render the PDF document, you have to setup a pixmap and draw on it.
fz_device structure: Represents the device on which the render result is drawn. Usually we create the device from the pixmap.

You don't have to care about the internal implementation of those structures. It means that when we write the P/Invoke code, we don't have to setup corresponding structs in our C# code. Using IntPtrs to hold references to them will do. Knowing this can considerably simplify our works in the P/Invoke part later.

With the knowledge of the above key structures, let's once more look into the fitz.h file and find out the following useful functions.

fz_new_context: Creates the fz_context.
fz_free_context: Frees the resources used by the fz_context.
fz_open_document_with_stream: Creates a fz_document instance out of a given fz_stream.
fz_open_file_w: Opens a fz_stream with a Unicode encoded file name. Notice: If you don't work with documents which have non ASCII characters in their file names, you can just use the fz_open_document function to open documents, instead of using the combination of fz_open_file_w and fz_open_document_with_stream.
fz_close_document: Closes the fz_document.
fz_close: Closes the fz_stream.
fz_count_pages: Gets the page numbers in a document.
fz_load_page: Creates a fz_page instance for a given page number.
fz_free_page: Frees the resources used by the fz_page.
fz_bound_page: Gets the dimension of a page.
fz_new_pixmap: Creates a fz_pixmap instance to hold the rendered visual result.
fz_clear_pixmap_with_value: Fills the fz_pixmap with a color (usually white).
fz_new_draw_device: Creates a fz_device to draw the rendered result.
fz_find_device_colorspace: Gets a fz_colorspace structure, used in the fz_run_page function.
fz_run_page: Renders the page to the specific fz_device.
fz_free_device: Frees the resources used by the fz_device.
fz_drop_pixmap: Frees the resources used by the fz_pixmap.
fz_pixmap_samples: Gets the data of the rendered fz_pixmap, used to render the Bitmap.

Writing the P/Invoke functions

Once we know about what functions we are going to use. We can start writing P/Invoke functions in C#. Firstly we begin with the following code in fitz.h for the fz_new_context function.

fz_context *fz_new_context(fz_alloc_context *alloc, fz_locks_context *locks, unsigned int max_store);

The fz_context * is the return value of the function, a pointer to a fz_context structure. In our P/Invoke function, IntPtr will be used since we don't care about the internal structure of fz_context.
fz_new_context is, of course, the name of the function.
fz_alloc_context * and fz_locks_context * are the types of the first two parameters of the function. When initializing the fz_context with the fz_new_context function, we can simply passing the IntPtr.Zero for those two parameters, hereby we will use IntPtr in the corresponding positions of the P/Invoke function.

Here is the P/Invoke function code for the fz_new_context function.

[DllImport (DLL, EntryPoint="fz_new_context")]
public static extern IntPtr NewContext (IntPtr alloc, IntPtr locks, uint max_store);

Let's do the same thing to the rest functions and write P/Invoke functions accordingly. During the course we will inevitably encounter three more new structures, fz_bbox, fz_rectangle and fz_matrix, referenced by those functions listed above. They are quite simple ones. We just define them as structs in our code, according to their definitions in fitz.h. Eventually we will reach something similar to the following code ready to be used by P/Invoke.

public struct BBox
{
    public int Left, Top, Right, Bottom;
}
public struct Rectangle
{
    public float Left, Top, Right, Bottom;
}
public struct Matrix
{
    public float A, B, C, D, E, F;
}
class NativeMethods {
 
    const string DLL = "libmupdf.dll";
 
    [DllImport (DLL, EntryPoint="fz_new_context")]
    public static extern IntPtr NewContext (IntPtr alloc, IntPtr locks, uint max_store);
 
    [DllImport (DLL, EntryPoint = "fz_free_context")]
    public static extern IntPtr FreeContext (IntPtr ctx);
 
    [DllImport (DLL, EntryPoint = "fz_open_file_w", CharSet = CharSet.Unicode)]
    public static extern IntPtr OpenFile (IntPtr ctx, string fileName);
 
    [DllImport (DLL, EntryPoint = "fz_open_document_with_stream")]
    public static extern IntPtr OpenDocumentStream (IntPtr ctx, string magic, IntPtr stm);
 
    [DllImport (DLL, EntryPoint = "fz_close")]
    public static extern IntPtr CloseStream (IntPtr stm);
 
    [DllImport (DLL, EntryPoint = "fz_close_document")]
    public static extern IntPtr CloseDocument (IntPtr doc);
 
    [DllImport (DLL, EntryPoint = "fz_count_pages")]
    public static extern int CountPages (IntPtr doc);
 
    [DllImport (DLL, EntryPoint = "fz_bound_page")]
    public static extern Rectangle BoundPage (IntPtr doc, IntPtr page);
 
    [DllImport (DLL, EntryPoint = "fz_clear_pixmap_with_value")]
    public static extern void ClearPixmap (IntPtr ctx, IntPtr pix, int byteValue);
 
    [DllImport (DLL, EntryPoint = "fz_find_device_colorspace")]
    public static extern IntPtr FindDeviceColorSpace (IntPtr ctx, string colorspace);
 
    [DllImport (DLL, EntryPoint = "fz_free_device")]
    public static extern void FreeDevice (IntPtr dev);
 
    [DllImport (DLL, EntryPoint = "fz_free_page")]
    public static extern void FreePage (IntPtr doc, IntPtr page);
 
    [DllImport (DLL, EntryPoint = "fz_load_page")]
    public static extern IntPtr LoadPage (IntPtr doc, int pageNumber);
 
    [DllImport (DLL, EntryPoint = "fz_new_draw_device")]
    public static extern IntPtr NewDrawDevice (IntPtr ctx, IntPtr pix);
 
    [DllImport (DLL, EntryPoint = "fz_new_pixmap")]
    public static extern IntPtr NewPixmap (IntPtr ctx, IntPtr colorspace, int width, int height);
 
    [DllImport (DLL, EntryPoint = "fz_run_page")]
    public static extern void RunPage (IntPtr doc, IntPtr page, IntPtr dev, Matrix transform, IntPtr cookie);
 
    [DllImport (DLL, EntryPoint = "fz_drop_pixmap")]
    public static extern void DropPixmap (IntPtr ctx, IntPtr pix);
 
    [DllImport (DLL, EntryPoint = "fz_pixmap_samples")]
    public static extern IntPtr GetSamples (IntPtr ctx, IntPtr pix);
 
}

Using the code - the flow of the program

The goal of this article is to render PDF documents into pictures. The procedure is quite simple.

Loads the document.
Iterates each page in the document.
Renders each page to Bitmap and saves them to the disk.
Releases allocated resources during the operation.

The code is listed below.

static void Main (string[] args) {
    const uint FZ_STORE_DEFAULT = 256 << 20;
    IntPtr ctx = NativeMethods.NewContext (IntPtr.Zero, IntPtr.Zero, FZ_STORE_DEFAULT); // Creates the context
    IntPtr stm = NativeMethods.OpenFile (ctx, "test.pdf"); // opens file test.pdf as a stream
    IntPtr doc = NativeMethods.OpenDocumentStream (ctx, ".pdf", stm); // opens the document
    int pn = NativeMethods.CountPages (doc); // gets the number of pages in the document
    for (int i = 0; i < pn; i++) { // iterate through each pages
        IntPtr p = NativeMethods.LoadPage (doc, i); // loads the page (first page number is 0)
        Rectangle b = NativeMethods.BoundPage (doc, p); // gets the page dimension
        using (var bmp = RenderPage (ctx, doc, p, b)) { // renders the page and converts the result to Bitmap
            bmp.Save ((i+1) + ".png"); // saves the bitmap to a file
        }
        NativeMethods.FreePage (doc, p); // releases the resources consumed by the page
    }
    NativeMethods.CloseDocument (doc); // releases the resources
    NativeMethods.CloseStream (stm);
    NativeMethods.FreeContext (ctx);
}

You can see that the flow of the above code is quite clean.

But... you may ask, "Wait a minute. How do you know that we can pass two IntPtr.Zeros and a FZ_STORE_DEFAULT to the NewContext function?" Good question. The answer is written in fitz.h--when you don't need to preallocate some memory nor set locks, you can simply use NULL, i.e. IntPtr.Zero, for the first two parameters; the value of FZ_STORE_DEFAULT was also mentioned there. So, please refer to fitz.h when you have questions.

What is left undone is that we have not yet written the code for the RenderPage function. We will finish it with the following lines of code.

static Bitmap RenderPage (IntPtr context, IntPtr document, IntPtr page, Rectangle pageBound) {
    Matrix ctm = new Matrix ();
    IntPtr pix = IntPtr.Zero;
    IntPtr dev = IntPtr.Zero;
 
    int width = (int)(pageBound.Right - pageBound.Left); // gets the size of the page
    int height = (int)(pageBound.Bottom - pageBound.Top);
    ctm.A = ctm.D = 1; // sets the matrix as the identity matrix (1,0,0,1,0,0)

    // creates a pixmap the same size as the width and height of the page
    pix = NativeMethods.NewPixmap (context, 
      NativeMethods.FindDeviceColorSpace (context, "DeviceRGB"), width, height);
    // sets white color as the background color of the pixmap
    NativeMethods.ClearPixmap (context, pix, 0xFF);
 
    // creates a drawing device
    dev = NativeMethods.NewDrawDevice (context, pix);
    // draws the page on the device created from the pixmap
    NativeMethods.RunPage (document, page, dev, ctm, IntPtr.Zero);
 
    NativeMethods.FreeDevice (dev); // frees the resources consumed by the device
    dev = IntPtr.Zero;
 
    // creates a colorful bitmap of the same size of the pixmap
    Bitmap bmp = new Bitmap (width, height, PixelFormat.Format24bppRgb); 
    var imageData = bmp.LockBits (new System.Drawing.Rectangle (0, 0, 
                      width, height), ImageLockMode.ReadWrite, bmp.PixelFormat);
    unsafe { // converts the pixmap data to Bitmap data
        // gets the rendered data from the pixmap
        byte* ptrSrc = (byte*)NativeMethods.GetSamples (context, pix);
        byte* ptrDest = (byte*)imageData.Scan0;
        for (int y = 0; y < height; y++) {
            byte* pl = ptrDest;
            byte* sl = ptrSrc;
            for (int x = 0; x < width; x++) {
                //Swap these here instead of in MuPDF because most pdf images will be rgb or cmyk.
                //Since we are going through the pixels one by one
                //anyway swap here to save a conversion from rgb to bgr.
                pl[2] = sl[0]; //b-r
                pl[1] = sl[1]; //g-g
                pl[0] = sl[2]; //r-b
                //sl[3] is the alpha channel, we will skip it here
                pl += 3;
                sl += 4;
            }
            ptrDest += imageData.Stride;
            ptrSrc += width * 4;
        }
    }
    NativeMethods.DropPixmap (context, pix);
    return bmp;
}

OK, everything is done. Just run the program and see each PDF pages in test.pdf converted into PNG files.

Points of Interest

There are quite a few problems you need to notice during practical development.

The first thing to notice is exceptions: documents may be corrupted, engaged, etc. Mupdf library does throw exceptions when such kinds of problems occur. However, since we are doing P/Invokes. All we can do is to catch the AccessViolationException, and rethrow specific, redefined exceptions, for example, PdfDocumentException can be retrown when we catch an exception while loading a document. The key is to find out when the exceptions will be thrown. You can find out information in the fitz.h file.

The second thing to notice is to remember releasing the resources. You may develop classes to encapsulate the creation and destroy of objects. For example, you may write a MupdfPage class to handle stuff about fz_page. Inside the constructor of MupdfPage, it creates the fz_page instance. The class should implement the IDisposable interface and put the P/Invoke code for fz_free_page in the Dispose method to release the resources.

The third thing to think about is to expand the functionalities of your wrapper. Currently the wrapper introduced by this article allows you to convert PDF pages into pictures. You may want to do more things and have more fun out of Mupdf. There're several ways:

Dig more out of the fitz.h. There are a lot more functions not covered by this article in that file. For example, you can find functions to open password protected documents (search for fz_needs_password and fz_authenticate_password).
Examining other header files, for example, mupdf.h file contains specific functions to examine and modify PDF files, mucbz.h allows you to view cbz files, etc.
Learn from examples and existing open-source projects. The official Mupdf website has provided several examples and the above C header files online. The three projects mentioned in this article are also good references for you to go further.

The fourth thing to consider is running on 64-bit machines. This is one of the most common issues when using P/Invoke. The DLL provided in the download of this article, compiled from the source code of SumatraPDF is 32-bit. It must be called from the 32-bit .NET Framework. On the 64-bit machines, the default .NET Framework will be the 64-bit one, which will fail when P/Invoking the 32-bit Mupdf DLL. Rather than recompiling the Mupdf DLL as a 64-bit DLL, you can instead change the CPU platform of the C# project from Any CPU to x86. Therefore, the compiled .NET program will be forced to run on the 32-bit .NET Framework and work well with the 32-bit Mupdf DLL.

The fifth thing to think about may be the basic of image processing. In our example, we render the PDF page with the default resolution. But what if we want it to be smaller (as page thumbnails) or bigger (for easier reading)? The "zoom" factor of the image lies in the Matrix structure that we pass into the RunPage function. If we modify its A and D number, the image will be scaled horizontally and vertically respectively (modifying other values will cause the image to be rotated, sheared or translated). If we set A or D to negative values, the rendered image will be flipped horizontally or vertically. By the way, don't forget to resize the dimension of the rendered pixmap and Bitmap to hold the resized image. Here's the code for your reference. You may probably program the zoomX and zoomY as parameters of the RenderPage function and replace the second code block in the original function with the rest lines of code below.

float zoomX = 1.0, zoomY = 1.0;

int width = (int)(zoomX * (pageBound.Right - pageBound.Left)); // gets the size of the scaled page
int height = (int)(zoomY * (pageBound.Bottom - pageBound.Top));
ctm.A = zoomX;
ctm.D = zoomY; // sets the matrix as (zoomX,0,0,zoomY,0,0)