Click here to Skip to main content
15,498,622 members
Articles / Mobile Apps / Android
Posted 10 Nov 2014

Tagged as


72 bookmarked

Android Character Recognition

Rate me:
Please Sign up or sign in to vote.
4.94/5 (42 votes)
10 Nov 2014CPOL4 min read
Optical Character Recognition with Android


In this article, i will present an OCR android demo application, that recognize words from a bitmap source.

There is an open source OCR library that supports android: Tesseract.

This demo project contains in addition other parts, like accessing the camera, handling bitmaps, making a camera focus box view, internal storage access etc.


OCR can be used in many purposes: reading text from images, scaning numbers or codes for specific services...

ContentImage 1

  1. Preparing Tesseract

  2. Adding tess-two to Android Studio Project

  3. Tesseract library usage

  4. Android implementation

Using the Code

The demo project in developed on a windows pc, using android studio IDE.

  1. Preparing Tesseract

  • Install the tesseract source-code from github
  • extract content in a tesseract folder
  • Requires Android 2.2 or higher
  • Download a v3.02 trained data file for a language (english data for example).
  • On the mobile side, data files must be extracted to a subdirectory named tessdata.

To import tesseract to your android project, yu must build it first:

  • You must have the android NDK, if you don't install it from here.
  • After installing the android ndk, you must add its install directory to the environement variables under Path
  • Go to Control Panel\System and Security\System - advanced system settings - environement variables:

Image 2

  • After adding the android directory to the path, we can use ndk command in the cmd.exe in other directory.
  • Now build the tesseract ocr library using the cmd window, (this process may take some time ~30 mins):
    • Go the the tess-two folder and open cmd window, (press Shift + Right Click):
      Image 3
    • Build the project using:
      Image 4
    • ndk-build
      android update project --path C:\...\tess-two
      ant release
  1. Adding Tess-Two to Android Studio Project

After we have build the tess-two library project, we must import it to the android application project in android studio.

  • In your android studio project tree, add a new directory "libraries", then add a subdirectory name it "tess-two".

Image 5

  • In windows explorer, move the content of the tess-two build project to the tess-two direcctory in libraries android studio.
  • You must add a build.gradle [new file] in the libraries\tess-two folder:
    - Make sure all build.gradle files in application project have same targetSdk version
    - Make sure that the tess-two library has build.gradle file
buildscript {
    repositories {
    dependencies {
        classpath ''

apply plugin: ''

android {

    compileSdkVersion 21
    buildToolsVersion "21.0.2"

    defaultConfig {
        minSdkVersion 15
        targetSdkVersion 21

    sourceSets.main {
        manifest.srcFile 'AndroidManifest.xml'
        java.srcDirs = ['src']
        resources.srcDirs = ['src']
        res.srcDirs = ['res']
        jniLibs.srcDirs = ['libs']
  • Next add to the main settings.gradle include the tess-two library:
    include ':app', ':tess-two'
    include ':libraries:tess-two'
    project(':tess-two').projectDir = new File('libraries/tess-two')
  • Next add the tess-two as module dependency to the app module in project structure(ctrl+alt+shift+s)

Image 6

  1. Now the tesseract library can be used in our android project:

public String detectText(Bitmap bitmap) {

    TessBaseAPI tessBaseAPI = new TessBaseAPI();

    String path = "/mnt/sdcard/packagename/tessdata/eng.traineddata";

    tessBaseAPI.init(path, "eng"); //Init the Tess with the trained data file, with english language

    //For example if we want to only detect numbers
    tessBaseAPI.setVariable(TessBaseAPI.VAR_CHAR_WHITELIST, "1234567890");
    tessBaseAPI.setVariable(TessBaseAPI.VAR_CHAR_BLACKLIST, "!@#$%^&*()_+=-qwertyuiop[]}{POIU" +


    String text = tessBaseAPI.getUTF8Text();

    Log.d(TAG, "Got data: " + result);

    return text;
  1. Android side

Still have to take a photo from the camera, or load a it from a file.
We will make a CameraEngine class that loads the camera hardware, and show live streaming on a SurfaceView.

In the CameraUtils:

//Check if the device has a camera
public static boolean deviceHasCamera(Context context) {
    return context.getPackageManager().hasSystemFeature(PackageManager.FEATURE_CAMERA);

//Get available camera
public static Camera getCamera() {
    try {
    } catch (Exception e) {
        Log.e(TAG, "Cannot getCamera()");
        return null;

In the CameraEngine:

public class CameraEngine {

    static final String TAG = "DBG_" + CameraUtils.class.getName();

    boolean on;
    Camera camera;
    SurfaceHolder surfaceHolder;

    Camera.AutoFocusCallback autoFocusCallback = new Camera.AutoFocusCallback() {
        public void onAutoFocus(boolean success, Camera camera) {


    public boolean isOn() {
        return on;

    private CameraEngine(SurfaceHolder surfaceHolder){
        this.surfaceHolder = surfaceHolder;

    static public CameraEngine New(SurfaceHolder surfaceHolder){
        Log.d(TAG, "Creating camera engine");
        return  new CameraEngine(surfaceHolder);

    public void requestFocus() {
        if (camera == null)

        if (isOn()) {

    public void start() {

        Log.d(TAG, "Entered CameraEngine - start()"); = CameraUtils.getCamera();

        if ( == null)

        Log.d(TAG, "Got camera hardware");

        try {

  ;//Portrait Camera

            on = true;

            Log.d(TAG, "CameraEngine preview started");

        } catch (IOException e) {
            Log.e(TAG, "Error in setPreviewDisplay");

    public void stop(){

        if(camera != null){
            camera = null;

        on = false;

        Log.d(TAG, "CameraEngine Stopped");

    public void takeShot(Camera.ShutterCallback shutterCallback,
                         Camera.PictureCallback rawPictureCallback,
                         Camera.PictureCallback jpegPictureCallback ){
            camera.takePicture(shutterCallback, rawPictureCallback, jpegPictureCallback);


Now in the MainActivity, we will have to:

  • Show the camera preview on a SurfaceView [On Resume]
  • Stop the camera preview and release the camera resource to let other apps use it. [On Pause]
  • Add two button: one for taking a shot (middle), another to focus(right).
  • Add a custom FocusBoxView to crop camera preview region, where text need to be extracted from.

The layout xml:

Image 7

<FrameLayout xmlns:android=""

        android:layout_height="fill_parent" />

        android:layout_height="fill_parent" />

        android:background="@drawable/shutter_layout" />

        android:background="@drawable/focus_layout" />


For the FocusBoxView, create class that extends View, we will need a Rect, that will represente to focus box, and change it dimension on the event, after that when the onDraw is called it will draw the focus box rectangle (design, frame, border and corners...) where the cropped photo will take place.

public class FocusBoxView extends View {

    private static final int MIN_FOCUS_BOX_WIDTH = 50;
    private static final int MIN_FOCUS_BOX_HEIGHT = 20;

    private final Paint paint;
    private final int maskColor;
    private final int frameColor;
    private final int cornerColor;

    public FocusBoxView(Context context, AttributeSet attrs) {
        super(context, attrs);

        paint = new Paint(Paint.ANTI_ALIAS_FLAG);
        Resources resources = getResources();

        maskColor = resources.getColor(R.color.focus_box_mask);
        frameColor = resources.getColor(R.color.focus_box_frame);
        cornerColor = resources.getColor(R.color.focus_box_corner);


    private Rect box;

    private static Point ScrRes;

    private  Rect getBoxRect() {

        if (box == null) {

            //FocusBoxUtils class contains some helper methods
            ScrRes = FocusBoxUtils.getScreenResolution(getContext());

            int width = ScrRes.x * 6 / 7;
            int height = ScrRes.y / 9;

            width = width == 0
                    ? MIN_FOCUS_BOX_WIDTH
                    : width < MIN_FOCUS_BOX_WIDTH ? MIN_FOCUS_BOX_WIDTH : width;

            height = height == 0
                    ? MIN_FOCUS_BOX_HEIGHT
                    : height < MIN_FOCUS_BOX_HEIGHT ? MIN_FOCUS_BOX_HEIGHT : height;

            int left = (ScrRes.x - width) / 2;
            int top = (ScrRes.y - height) / 2;

            box = new Rect(left, top, left + width, top + height);

        return box;

    public Rect getBox() {
        return box;

    private void updateBoxRect(int dW, int dH) {


    private OnTouchListener touchListener;

    private OnTouchListener getTouchListener() {

        if (touchListener == null)
            touchListener = new OnTouchListener() {

                int lastX = -1;
                int lastY = -1;

                public boolean onTouch(View v, MotionEvent event) {
                    switch (event.getAction()) {
                        case MotionEvent.ACTION_DOWN:
                            lastX = -1;
                            lastY = -1;
                            return true;
                        case MotionEvent.ACTION_MOVE:
                            int currentX = (int) event.getX();
                            int currentY = (int) event.getY();
                            try {
                                 ... updateBoxRect(dx, dy);
                            } catch (NullPointerException e) {

                            return true;
                        case MotionEvent.ACTION_UP:
                            lastX = -1;
                            lastY = -1;
                            return true;
                    return false;

        return touchListener;

    public void onDraw(Canvas canvas) {

        Rect frame = getBoxRect();

        int width = canvas.getWidth();
        int height = canvas.getHeight();

        .... DRAW FOCUS BOX

        canvas.drawCircle(frame.left - 32, - 32, 32, paint);
        canvas.drawCircle(frame.right + 32, - 32, 32, paint);
        canvas.drawCircle(frame.left - 32, frame.bottom + 32, 32, paint);
        canvas.drawCircle(frame.right + 32, frame.bottom + 32, 32, paint);

Image 8

Note that you must add in the AndroidManifest.xml permission to use the camera, and other used features:

<uses-permission android:name="android.permission.CAMERA"/>
   <uses-feature android:name="" />
       android:required="false" />
   <uses-feature android:name="" />

Now let's return to the MainActivity, when the focus button is clicked, we will request a focus from the camera,

when the camera button is clicked, the camera will take a photo, and callback the onPictureTaken(byte[] data, Camera camera) where we will decode the byte array to bitmap and resize, perform the image crop in Tools.getFocusedBitmap(this, camera, data, focusBox.getBox()), and call the TesseractBaseApi under the Async class TessAsyncEngine to extract and show a dialog that holds the text and show cropped photo.

For your custom use, you will change or update you code given your needs.

public class MainActivity extends Activity implements SurfaceHolder.Callback, View.OnClickListener,
        Camera.PictureCallback, Camera.ShutterCallback {

    static final String TAG = "DBG_" + MainActivity.class.getName();

    Button shutterButton;
    Button focusButton;
    FocusBoxView focusBox;
    SurfaceView cameraFrame;
    CameraEngine cameraEngine;

    protected void onCreate(Bundle savedInstanceState) {

    public void surfaceCreated(SurfaceHolder holder) {

        Log.d(TAG, "Surface Created - starting camera");

        if (cameraEngine != null && !cameraEngine.isOn()) {

        if (cameraEngine != null && cameraEngine.isOn()) {
            Log.d(TAG, "Camera engine already on");

        cameraEngine = CameraEngine.New(holder);

        Log.d(TAG, "Camera engine started");

    public void surfaceChanged(SurfaceHolder holder, int format, int width, int height) {


    public void surfaceDestroyed(SurfaceHolder holder) {


    protected void onResume() {

        cameraFrame = (SurfaceView) findViewById(;
        shutterButton = (Button) findViewById(;
        focusBox = (FocusBoxView) findViewById(;
        focusButton = (Button) findViewById(;


        SurfaceHolder surfaceHolder = cameraFrame.getHolder();


    protected void onPause() {

        if (cameraEngine != null && cameraEngine.isOn()) {

        SurfaceHolder surfaceHolder = cameraFrame.getHolder();

    public void onClick(View v) {
        if(v == shutterButton){
            if(cameraEngine != null && cameraEngine.isOn()){
                cameraEngine.takeShot(this, this, this);

        if(v == focusButton){
            if(cameraEngine!=null && cameraEngine.isOn()){

    public void onPictureTaken(byte[] data, Camera camera) {

        Log.d(TAG, "Picture taken");

        if (data == null) {
            Log.d(TAG, "Got null data");

        Bitmap bmp = Tools.getFocusedBitmap(this, camera, data, focusBox.getBox());

        Log.d(TAG, "Got bitmap");

        new TessAsyncEngine().executeOnExecutor(AsyncTask.SERIAL_EXECUTOR, this, bmp);


    public void onShutter() {


For bitmap crop in Imaging.Tools class:

public static Bitmap getFocusedBitmap(Context context, Camera camera, byte[] data, Rect box){
       Point CamRes = FocusBoxUtils.getCameraResolution(context, camera);
       Point ScrRes = FocusBoxUtils.getScreenResolution(context);

       int SW = ScrRes.x; //SCREEN WIDTH - HEIGHT
       int SH = ScrRes.y;

       int RW = box.width(); // FOCUS BOX RECT WIDTH - HEIGHT - TOP - LEFT
       int RH = box.height();
       int RL = box.left;
       int RT =;

       float RSW = (float) (RW * Math.pow(SW, -1)); //DIMENSION RATIO OF FOCUSBOX OVER SCREEN
       float RSH = (float) (RH * Math.pow(SH, -1));

       float RSL = (float) (RL * Math.pow(SW, -1));
       float RST = (float) (RT * Math.pow(SH, -1));

       float k = 0.5f;

       int CW = CamRes.x;
       int CH = CamRes.y;

       int X = (int) (k * CW); //SCALED BITMAP FROM CAMERA
       int Y = (int) (k * CH);


       Bitmap unscaledBitmap = Tools.decodeByteArray(data, X, Y, Tools.ScalingLogic.CROP);
       Bitmap bmp = Tools.createScaledBitmap(unscaledBitmap, X, Y, Tools.ScalingLogic.CROP);

       if (CW > CH)
           bmp = Tools.rotateBitmap(bmp, 90);

       int BW = bmp.getWidth();   //NEW FULL CAPTURED BITMAP DIMENSIONS
       int BH = bmp.getHeight();

       int RBT = (int) (RST * BH);

       int RBW = (int) (RSW * BW);
       int RBH = (int) (RSH * BH);

       Bitmap res = Bitmap.createBitmap(bmp, RBL, RBT, RBW, RBH);

       return res;

In the end here is a result photo:

Image 9

Points of Interest

If you are interested in using OCR engines, i hope this simple article will help you. Thanks.


This article, along with any associated source code and files, is licensed under The Code Project Open License (CPOL)

Written By
Lebanon Lebanon
Bitcoin, Ethereum - Smartcontracts, Full Stack, Architecture & Development, Music!

Comments and Discussions

QuestionApp Crashing on MainActivity Pin
Member 1420086828-Mar-19 10:06
MemberMember 1420086828-Mar-19 10:06 
QuestionCMD STEP 1 ERROR: Your APP_BUILD_SCRIPT points to an unknown file: ./jni/ Pin
yuh1ra13-Aug-18 6:09
Memberyuh1ra13-Aug-18 6:09 
Questiongradle error Pin
Member 137030556-Mar-18 4:44
MemberMember 137030556-Mar-18 4:44 
Questionsize of box Pin
Member 1248265424-Jul-17 19:55
MemberMember 1248265424-Jul-17 19:55 
QuestionHow can a flashlight be added to this? Pin
Vi1217-Jun-17 0:57
MemberVi1217-Jun-17 0:57 
GeneralMy vote of 5 Pin
Member 1312635813-Apr-17 5:10
MemberMember 1312635813-Apr-17 5:10 
QuestionError when trying to add tess-two as module dependency to the app Pin
Member 130318551-Mar-17 9:48
MemberMember 130318551-Mar-17 9:48 
QuestionCould not recognize TessDataManager class for running example Pin
Member 1302107524-Feb-17 1:48
MemberMember 1302107524-Feb-17 1:48 
AnswerRe: Could not recognize TessDataManager class for running example Pin
Member 1302192324-Feb-17 7:30
MemberMember 1302192324-Feb-17 7:30 
QuestionQ? Pin
jhon kifle8-Feb-17 21:00
professionaljhon kifle8-Feb-17 21:00 
AnswerRe: Q? Pin
F. Aro9-Feb-17 21:45
professionalF. Aro9-Feb-17 21:45 
Questioncharacter recognition issue Pin
Azizullah Samim6-Nov-16 20:29
MemberAzizullah Samim6-Nov-16 20:29 
Questionerror Pin
AJay Kumar27-Oct-16 2:42
MemberAJay Kumar27-Oct-16 2:42 
Questionpercentage progress values Pin
Joseph Francis Sarr9-Sep-16 5:57
MemberJoseph Francis Sarr9-Sep-16 5:57 
GeneralMy vote of 5 Pin
Muhammad Shahid Farooq30-Aug-16 23:48
professionalMuhammad Shahid Farooq30-Aug-16 23:48 
GeneralRe: My vote of 5 Pin
F. Aro30-Aug-16 23:50
professionalF. Aro30-Aug-16 23:50 
QuestionAndroid OCR Pin
AdeleB2-Aug-16 22:11
MemberAdeleB2-Aug-16 22:11 
AnswerRe: Android OCR Pin
F. Aro30-Aug-16 23:50
professionalF. Aro30-Aug-16 23:50 
Questionstep 2 Pin
Member 1263118912-Jul-16 5:40
MemberMember 1263118912-Jul-16 5:40 
QuestionHelp on Step 2. Pin
Member 1259430420-Jun-16 19:07
MemberMember 1259430420-Jun-16 19:07 
QuestionCapture consecutively Pin
Thái Sơn Hoàng2-Jun-16 18:31
MemberThái Sơn Hoàng2-Jun-16 18:31 
Questionimproving accuracy Pin
jp140410-Mar-16 22:04
Memberjp140410-Mar-16 22:04 
AnswerRe: improving accuracy Pin
Member 99626387-Apr-16 22:14
MemberMember 99626387-Apr-16 22:14 
QuestiondetectText have no influence Pin
Member 1235863129-Feb-16 3:37
MemberMember 1235863129-Feb-16 3:37 
AnswerRe: detectText have no influence Pin
Member 1235863129-Feb-16 5:16
MemberMember 1235863129-Feb-16 5:16 

General General    News News    Suggestion Suggestion    Question Question    Bug Bug    Answer Answer    Joke Joke    Praise Praise    Rant Rant    Admin Admin   

Use Ctrl+Left/Right to switch messages, Ctrl+Up/Down to switch threads, Ctrl+Shift+Left/Right to switch pages.