Click here to Skip to main content
Click here to Skip to main content
Go to top

AndroidVision: Learn Image Processing on your mobile

, 20 Jan 2014
Rate this:
Please Sign up or sign in to vote.
Experiment with OpenCV imageprocessing methods on your mobile phone

Contents 

Demo

Introduction

This article describes an Android application demonstrating the capabilities of the OpenCV platform on Android. Its main goal is NOT speed of execution but ease of implementation. After all, I want to demonstrate the outcome of using specific filters and as such, there is no optimization.

And as always: I'm innocent:

This is the "get it out now" release. This means the infrastructure is there but the supported methods and configuration are basic. However, look at it positively: this also means that there is room for improvement!

What do you need?

Allthough I use three libraries, you will only need to download one: the OpenCV library. I'm using version 2.4.5 . The other two libraries are included in the source download. They are my own Touch DSL and Object editor. There use in this application will be explained, but for their inner workings you'll have to read the articles.

All code was written with Andoid 3.0 in mind.

Main Activity and capturing the camera

The Structure

The starting activity is the AndroidVision activity which is heavily based on the Tutorial2Activity activity in the OpenCV Tutorial 2 - Mixed Processing sample code for Android. The view FrameGrabberView is also mostly a copy of CameraBridgeViewBase in the OpenCV library. The changes made are actually in FrameGrabberViewBase to which I added the capability to register a FrameGrabberViewBase.CvCameraViewDrawable allowing you to draw stuff on top of the cameraview.

One of the changes to the AndroidVision activity are that it now implements three menu options:

  1. Select: allowing you to select an imageprocessing method.
  2. Configure: allowing you to configure the selected method.
  3. Show: shows the configuration of the method in the screen.

The Code

AndroidVision

The basic functions of the activity are:

  • create a, currently only backfacing, cameraview
  • register for the CvCameraViewListener2 events
  • initialize the OpenCV library and subsequently the cameraview
  • register for the FrameGrabberViewBase.CvCameraViewDrawable events
  • apply the selected IImageProcessingMethod

Creation of the cameraview is done in the OnCreate method of the AndroidVision activity

@Override
public void onCreate(Bundle savedInstanceState) {
	super.onCreate(savedInstanceState);
	
	// More code ...
	
	View cameraView = CreateBackFacingCameraView();
	cameraView.setOnTouchListener(this);
	
	setContentView(cameraView);       

}

private View CreateBackFacingCameraView()
{
	LayoutInflater inf = (LayoutInflater) this.getSystemService(Context.LAYOUT_INFLATER_SERVICE);
	
	View cameraView = inf.inflate(R.layout.framegrabber_view, null);
	
	mCameraFacingBackView = (FrameGrabberViewBase)cameraView.findViewById(R.id.fgView);
	
	int cameraCount = 0;

	CameraInfo cameraInfo = new CameraInfo();
	cameraCount = Camera.getNumberOfCameras();
	int cameraIndex = 0;
	for ( int camIdx = 0; camIdx < cameraCount; camIdx++ ) {
		Camera.getCameraInfo( camIdx, cameraInfo );
		if ( cameraInfo.facing == Camera.CameraInfo.CAMERA_FACING_BACK  ) {
			cameraIndex = camIdx;
			break;
		}
	}
	
	mCameraFacingBackView.setCameraIndex(cameraIndex);
	
	mCameraFacingBackView.setVisibility(SurfaceView.VISIBLE);
	mCameraFacingBackView.setCvCameraViewListener(this);
	
	return cameraView;
}

Registering for the FrameGrabberViewBase.CvCameraViewListener2 events, which deliver the frames on which the processing is performed, is done during creation of the backfacing camera view, see CreateBackFacingCameraView()

This is the FrameGrabberViewBase.CvCameraViewListener2 interface

public abstract class FrameGrabberViewBase extends SurfaceView implements SurfaceHolder.Callback 
{
	// More code...
	
	public interface CvCameraViewListener2 {
		public void onCameraViewStarted(int width, int height);
		public void onCameraViewStopped();
		public Mat onCameraFrame(CvCameraViewFrame inputFrame);
	};
	
	// More code
}

Initialisation of OpenCV is done in the OnResume which registers a callback which, after initialization of OpenCV, initialises the view.

private BaseLoaderCallback mLoaderCallback = new BaseLoaderCallback(this) {
	@Override
	public void onManagerConnected(int status) {
		switch (status) {
			case LoaderCallbackInterface.SUCCESS:
			{            		
				Log.i(TAG, "OpenCV loaded successfully");
				InitializeView();
			} break;
			default:
			{
				super.onManagerConnected(status);
			} break;
		}
	}
};
    
@Override
public void onResume()
{
	super.onResume();
	OpenCVLoader.initAsync(OpenCVLoader.OPENCV_VERSION_2_4_3, this, mLoaderCallback);
}

private void InitializeView()
{
	InitializeBackfacingCameraView();			
	
	ApplyMethodOnCurentView();
}

private void InitializeBackfacingCameraView()
{
	if (mCameraFacingBackView != null)
	{	
		mCameraFacingBackView.setCameraViewDrawable(overlayList);
		mCameraFacingBackView.enableView();
	}
}

Registering for FrameGrabberViewBase.CvCameraViewDrawable events of FramegrabberView, which allows you to draw on the Canvas of the view, is done in InitializeBackfacingCameraView of which the code is shown above.

And this is the interface:

public abstract class FrameGrabberViewBase extends SurfaceView implements SurfaceHolder.Callback
{
	// More code...
	
    public interface CvCameraViewDrawable {
    	public void setResolution(int width, int height);   	
    	public void draw(Canvas canvas);
    }
	
	// More code
}

There are two objects which draw stuff in the Framegrabberview:

  1. ImageProcessingMethod which can draw its configuration settings
  2. ObjectEditorCameraViewDrawable which can draw controls to edit the configuration settings

What they do exactly will be explained further.

Both are abstracted through the CameraViewDrawableList

public class CameraViewDrawableList implements FrameGrabberViewBase.CvCameraViewDrawable {

	public void setImageMethodOverlay(FrameGrabberViewBase.CvCameraViewDrawable imageMethod)
	{
		if(Contains(this.imageMethod))
		{
			Remove(this.imageMethod);
		}
		this.imageMethod = imageMethod;
		Add(this.imageMethod);
	}

	public void setObjectEditorOverlay(FrameGrabberViewBase.CvCameraViewDrawable objectEditor)
	{
		if(Contains(objectEditor))
		{
			Remove(objectEditor);
		}
		this.objectEditor = objectEditor;
		Add(this.objectEditor);
	}
	
	private void Add(FrameGrabberViewBase.CvCameraViewDrawable item)
	{
		if(width != 0 && height != 0)
		{
			item.setResolution(width, height);
		}
		
		cameraViewDrawableList.add(item);
	}
	
	private void Remove(FrameGrabberViewBase.CvCameraViewDrawable item)
	{
		cameraViewDrawableList.remove(item);
	}
	
	private boolean Contains(FrameGrabberViewBase.CvCameraViewDrawable item)
	{
		return cameraViewDrawableList.contains(item);
	}

	@Override
	public void setResolution(int width, int height) {
		this.width = width;
		this.height = height;
		
		for(FrameGrabberViewBase.CvCameraViewDrawable item : cameraViewDrawableList)
		{
			item.setResolution(width, height);
		}
	}

	@Override
	public void draw(Canvas canvas) {
		if(objectEditor != null)
			objectEditor.draw(canvas);
		if(imageMethod != null)
			imageMethod.draw(canvas);
	}
	
	private int width = 0;
	private int height = 0;
	
	private FrameGrabberViewBase.CvCameraViewDrawable imageMethod;
	private FrameGrabberViewBase.CvCameraViewDrawable objectEditor;
	private ArrayList<FrameGrabberViewBase.CvCameraViewDrawable> cameraViewDrawableList = new ArrayList<FrameGrabberViewBase.CvCameraViewDrawable>();
}

Finally, the selected IImageProcessingMethod is applied in the onCameraFrame callback method: (see the FrameGrabberViewBase.CvCameraViewListener2 interface)

@Override
public Mat onCameraFrame(CvCameraViewFrame inputFrame) {
	mInputMat = inputFrame.rgba();
	return ApplyMethod(mInputMat);
}

private Mat ApplyMethod(Mat mRgba)
{
	if(mMethod != null && mRgba != null)
	{
		Mat sourceImage = new Mat();
		
		// Some methods only apply to a grayscale image, so we must first convert out RGBA image
		if(mMethod.getInputImageType() == ImageType.Gray)
		{
			Imgproc.cvtColor(mRgba, sourceImage, Imgproc.COLOR_BGRA2GRAY, 4);
		}
		else
		{
			sourceImage = mRgba;
		}
		
		if(mResultMat == null)
		{
			mResultMat = new Mat();
		}
		
		mMethod.setInputImage(sourceImage);
		mMethod.setOutputImage(mResultMat);
		
		mMethod.Execute();
		
		mResultMat = mMethod.getOutputImage();
	}

	return mResultMat;
}

FrameGrabberViewBase

Following code shows only the part which enables drawing on top of the camera view.

public abstract class FrameGrabberViewBase extends SurfaceView implements SurfaceHolder.Callback 
{
	// More methods ...
	
    protected void deliverAndDrawFrame(CvCameraViewFrame frame) {
		// More code which calls onCameraFrame, 
		//	converts the returned Mat into a Bitmap 
		//	and draws it in the surfaceholder
	
				// Perform your custom drawing in the Canvas of the surfaceholder
				// This drawing is done after the the Bitmap with the OnCameraFrame result is drawn, 
				//	thus resulting in us drawing on top of the edited image
                if (mCameraViewDrawable != null) {
                    mCameraViewDrawable.draw(canvas);
                }
	}
	
	// More methods ...
}

Selecting, Configuring and Applying methods

The Structure

The supported method names are stored in the AvailableImageProcessingMethods class which also categorizes the methods. Selection happens in the ImageProcessingMethodSelection Activity which has a ExpandableListView as a view. By supplying an object of type AvailableImageProcessingMethods to AvailableImageMethodAdapter we get a categorized view of available methods.

After selecting a method it can also be configured. This is done in the ImageProcessingMethodConfiguration which has a ObjectEditorView for editing of the configuration properties. During editing of the configuration properties you can also select properties to be editable in the cameraview. This will be explained in the next section.

Finally the selected IImageProcessingMethod is applied on the frames supplied by the camera.

The Code

ImageProcessingMethodSelection

public class ImageProcessingMethodSelection extends Activity implements OnChildClickListener, OnGroupClickListener {
	private static String SELECTED_METHOD = "METHODSELECTOR_SELECTED_METHOD";
	
	AvailableImageMethodAdapter adapter;
	ExpandableListView availableImageMethodView;
 
	@Override
	protected void onCreate(Bundle savedInstanceState) {
		super.onCreate(savedInstanceState);
		setContentView(R.layout.imagemethod_selection);

		adapter = new AvailableImageMethodAdapter(this, new AvailableImageProcessingMethods());

		availableImageMethodView = (ExpandableListView) findViewById(R.id.exlstImgMethSelect);
		availableImageMethodView.setAdapter(adapter);
		availableImageMethodView.setGroupIndicator(null);
		availableImageMethodView.setOnChildClickListener(this);
		availableImageMethodView.setOnGroupClickListener(this);

		int numberOfGroups = adapter.getGroupCount();
		for (int i = 0; i < numberOfGroups; i++)
		{
			availableImageMethodView.expandGroup(i);
		}
	}

	public static String getSelectedMethod(Bundle bundle)
	{
		if(!bundle.containsKey(SELECTED_METHOD))
		{
			return null;
		}
		return bundle.getString(SELECTED_METHOD);
	}
	
	public static void setSelectedMethod(Bundle bundle, String selectedMethod)
	{
		bundle.putString(SELECTED_METHOD, selectedMethod);
	}

	@Override
	public boolean onChildClick(ExpandableListView parent, View v,
			int groupPosition, int childPosition, long id) {
    	String selectedMethodId = (String)adapter.getChild(groupPosition, childPosition);
	    Intent result = new Intent();
	    Bundle bundle = new Bundle();
	    setSelectedMethod(bundle, selectedMethodId);
	    result.putExtras(bundle);

	    setResult(Activity.RESULT_OK, result);
	    
	    super.onBackPressed();
        return false;
	}

	@Override
	public boolean onGroupClick(ExpandableListView arg0, View arg1, int arg2,
			long arg3) {
		// prevent the group from collapsing
		return true;
	}
}

As you can see, the available methods are abstracted in the AvailableImageProcessingMethods class

public class AvailableImageProcessingMethods {
	private ArrayList<String> availableCategories = new ArrayList<String>();
	private Map<String, ArrayList<String>> categoriesMethodIdMap = new Hashtable<String, ArrayList<String>>(); 
	private Map<String, IImageProcessingMethod> methodIdToMethodMap = new Hashtable<String, IImageProcessingMethod>(); 
	private Map<String, String> methodIdToDisplayNameMap = new Hashtable<String, String>(); 
	
	public AvailableImageProcessingMethods()
	{
		RegisterMethod("default", "None", new NoOpMethod());
		RegisterMethod("blur", "Box", new BoxBlur());
		RegisterMethod("blur", "Gaussian", new GaussianBlur());
		RegisterMethod("blur", "Median", new MedianBlur());
		RegisterMethod("edge detection", "Canny", new CannyEdgeDetection());
		RegisterMethod("edge detection", "Sobel", new SobelEdgeDetection());
		RegisterMethod("hough transform", "Lines", new HoughLines());
		RegisterMethod("hough transform", "Circles", new HoughCircles());
		RegisterMethod("mathematical morph.", "Erosion", new ErosionMathMorph());
		RegisterMethod("mathematical morph.", "Dilation", new DilationMathMorph());
		RegisterMethod("various", "Histigram Eq.", new HistogramEqualization());
	}
	
	public ArrayList<String> getAvailableCategories()
	{
		return availableCategories;
	}
	
	public String getCategory(int index)
	{
		return availableCategories.get(index);
	}
	
	public ArrayList<String> getAvailableMethodIds(String category) {
		return categoriesMethodIdMap.get(category);
	}
	
	public IImageProcessingMethod getMethod(String methodId)
	{
		if(!methodIdToMethodMap.containsKey(methodId))
		{
			return null;
		}
		return methodIdToMethodMap.get(methodId);
	}
	
	public String getMethodDisplayname(String methodId)
	{
		return methodIdToDisplayNameMap.get(methodId);
	}
	
	private void RegisterMethod(String category, String displayName, IImageProcessingMethod method)
	{
		if(!availableCategories.contains(category))
		{
			availableCategories.add(category);
		}
		if(!categoriesMethodIdMap.containsKey(category))
		{
			categoriesMethodIdMap.put(category, new ArrayList<String>());
		}
		categoriesMethodIdMap.get(category).add(method.getMethodId());
		methodIdToMethodMap.put(method.getMethodId(), method);
		methodIdToDisplayNameMap.put(method.getMethodId(), displayName);
	}
}

The AvailableImageMethodAdapter extends BaseExpandableListAdapter so that the available methods show up as an expandable list:

public class AvailableImageMethodAdapter extends BaseExpandableListAdapter {

	// More code ...
	
	@Override
	public View getChildView(int groupPosition, int childPosition,
			boolean isLastChild, View convertView, ViewGroup parent) {
		
        String imgMethodId = (String) getChild(groupPosition, childPosition);
        
        if (convertView == null) {
            LayoutInflater infalInflater = (LayoutInflater) context
                    .getSystemService(Context.LAYOUT_INFLATER_SERVICE);
            convertView = infalInflater.inflate(R.layout.imagemethod_item, null);
        }
 
        TextView tvImageMethodId = (TextView)convertView.findViewById(R.id.tvImgMethItem); 
        tvImageMethodId.setText(availableMethods.getMethodDisplayname(imgMethodId));
        
        return convertView;
	}

	// More code ...

	@Override
	public View getGroupView(int groupPosition, boolean isExpanded,
			View convertView, ViewGroup parent) {
		
		String category = (String) getGroup(groupPosition);
		
		if (convertView == null) {
			LayoutInflater inf = (LayoutInflater) context.getSystemService(Context.LAYOUT_INFLATER_SERVICE);
			convertView = inf.inflate(R.layout.imagemethod_group, null);
		}
		
		TextView tvCategory = (TextView)convertView.findViewById(R.id.tvImgMethGrp);
		tvCategory.setText(category);

		return convertView;
	}
	
	// More code ...
	
}

Starting the ImageProcessingMethodSelection activity and processing its result are done in AndroidVision

private void handleSelectMethod()
{
	Intent myIntent = new Intent(AndroidVision.this, ImageProcessingMethodSelection.class);
	startActivityForResult(myIntent, REQUEST_METHODSELECTOR);
}

private void handleSelectMethodResult(int requestCode, int resultCode, Intent intent)
{
	if(intent == null)
	{
		return;
	}
	
	String requestedMethodId = ImageProcessingMethodSelection.getSelectedMethod(intent.getExtras());
	
	if(requestedMethodId == null || requestedMethodId.length() == 0)
	{
		return;
	}
	selectMethod(requestedMethodId);

}

private void selectMethod(String methodId)
{
	IImageProcessingMethod requestedMethod = availableMethods.getMethod(methodId);
	if(requestedMethod == null)
	{
		return;
	}
	
	mMethod = requestedMethod;

	currentMethod = methodId;
	
	overlayList.setImageMethodOverlay(mMethod);
	
	ApplyConfigurationOnObjectEditor();
	
	ApplyMethodOnCurentView();		
}

private void ApplyConfigurationOnObjectEditor()
{
	if(mMethod == null)
	{
		return;
	}
	
	IImageProcessingMethodConfiguration configuration = mMethod.getConfiguration();
	
	ArrayList<PropertyProxy> inViewEditableProps = new ArrayList<PropertyProxy>();
	if(configuration != null)
	{
		for(String propertyName : configuration.availablePropertyList())
		{
			inViewEditableProps.add(PropertyProxy.CreateFomPopertyName(propertyName, configuration, converterRegistry));
		}
	}
	
	propertyEditorOverlay.setPropertyList(inViewEditableProps);
}

ImageProcessingMethodConfiguration

Configuration is done using my Object editor library. As you can see below in the onCreate method, the frames in which the editors are shown are customized allowing to provide a small link icon in the top-right corner. Using this link one can make properties available for direct editing in the cameraview. By registering for the creation events of the object editor we can attach a click handler to these icons.

public class ImageProcessingMethodConfiguration extends Activity implements ItemCreationCallback, OnClickListener {
	public  static String CONFIGURATION = "METHODCONFIGURATION_CONFIGURATION";

	private ObjectProxy objectProxy;
	private ObjectEditorView view;
	private IImageProcessingMethodConfiguration objectToEdit = null;

	@Override
	public void onCreate(Bundle savedInstanceState) {
		super.onCreate(savedInstanceState);

		ObjectEditorViewConfig config = new ObjectEditorViewConfig();
		config.FrameId = R.layout.imageproperty_frame;
		config.FrameNameId = R.id.tvPropertyName;
		config.FramePlaceholderId = R.id.llPropertyViewHolder;

		config.FrameDlgId = R.layout.imageproperty_external_frame;
		config.FrameDlgNameId = R.id.tvPropertyNameExt;
		config.FrameDlgPlaceholderId = R.id.llPropertyViewHolderExt;

		config.FrameIntentId = R.layout.imageproperty_external_frame;
		config.FrameIntentNameId = R.id.tvPropertyNameExt;
		config.FrameIntentPlaceholderId = R.id.llPropertyViewHolderExt;

		view = new ObjectEditorView(this, config);

		view.setItemCreationCallback(this);

		Bundle b = getIntent().getExtras(); 
		objectToEdit = (IImageProcessingMethodConfiguration)b.getParcelable(CONFIGURATION); 

		TypeConverterRegistry converterRegistry = TypeConverterRegistry.create();
		objectProxy = new ObjectProxy(objectToEdit, converterRegistry);
		view.setObjectToEdit(objectProxy);

		setContentView(view);
	}


	@Override
	public void onBackPressed() {
		Intent result = new Intent();
		result.putExtra(ImageProcessingMethodConfiguration.CONFIGURATION, (Parcelable)objectToEdit);

		setResult(Activity.RESULT_OK, result);

		super.onBackPressed();
	}

	@Override
	protected void onDestroy() {
		super.onDestroy();

		objectProxy.Destroy();
		objectProxy = null;
	}

	@Override
	public void OnFrameCreated(View frame) {
		View imageFrame = frame.findViewById(R.id.ivPropertyConnected);
		if(imageFrame != null)
		{
			imageFrame.setOnClickListener(this);
			return;
		}
		View imageFrameExt = frame.findViewById(R.id.ivPropertyConnectedExt);
		if(imageFrameExt != null)
		{
			imageFrameExt.setOnClickListener(this);
			return;
		}
	}

	@Override
	public void OnFrameInitialized(View frame, PropertyProxy property) {
		ImageView imageFrame = (ImageView)frame.findViewById(R.id.ivPropertyConnected);
		String propertyName = property.getName();
		if(imageFrame == null)
		{
			imageFrame = (ImageView)frame.findViewById(R.id.ivPropertyConnectedExt);
		}
		if(imageFrame != null)
		{
			if(objectToEdit.isPropertyAllowedInView(propertyName))
			{
				setEditInViewIcon(imageFrame, propertyName);
				imageFrame.setTag(property.getName());
			}
			return;
		}
	}

	@Override
	public void OnFrameUnInitialized(View frame) {
		View imageFrame = frame.findViewById(R.id.ivPropertyConnected);
		if(imageFrame == null)
		{
			imageFrame = frame.findViewById(R.id.ivPropertyConnectedExt);
		}
		if(imageFrame != null)
		{
			imageFrame.setTag(null);
			return;
		}
	}

	@Override
	public void onClick(View view) {
		ImageView imageFrame = (ImageView)view;
		if (imageFrame != null)
		{
			String propertyName = (String)imageFrame.getTag();
			toggleEditInView(propertyName);
			setEditInViewIcon(imageFrame, propertyName);
		}
	}
	
	private void toggleEditInView(String propertyName)
	{
		if(objectToEdit.isPropertyAvailableInView(propertyName))
		{
			objectToEdit.removePropertyToAvailableInView(propertyName);
		}
		else
		{
			objectToEdit.addPropertyToAvailableInView(propertyName);
		}
	}
	
	private void setEditInViewIcon(ImageView imageFrame, String propertyName)
	{
		if(objectToEdit != null && propertyName != null)
		{
			if(objectToEdit.isPropertyAvailableInView(propertyName))
			{
				imageFrame.setImageResource(R.drawable.connected);
			}
			else
			{
				imageFrame.setImageResource(R.drawable.disconnected);
			}
		}
	}
}

Configuration objects are handed to the editor by using parcelables. For this, all configuration classes implement Parcelable

public class BoxConfiguration extends ImageProcessingMethodConfigBase implements Parcelable, IImageProcessingMethodConfiguration   {

	int kernelSize = 1;
	
	public BoxConfiguration()
	{
		fillInViewEditingAllowed(this);
	}
	
	public BoxConfiguration(Parcel in)
	{
		fillInViewEditingAllowed(this);
		readFromParcel(in);
	}
	
	// More methods unimportant to the Parcelable implementation
	
	@Override
	public int describeContents() {
		return 0;
	}
	
	public void readFromParcel(Parcel in) {
		baseReadFromParcel(in);
		kernelSize = in.readInt();
	}

	@Override
	public void writeToParcel(Parcel arg0, int arg1) {
		baseWriteToParcel(arg0, arg1);
		arg0.writeInt(kernelSize);

	}
	
	public static final Parcelable.Creator<BoxConfiguration> CREATOR = new Parcelable.Creator<BoxConfiguration>()
	{
		@Override
		public BoxConfiguration createFromParcel(Parcel source) {
			return new BoxConfiguration(source);
		}

		@Override
		public BoxConfiguration[] newArray(int size) {
			return new BoxConfiguration[size];
		}
	};
}

Starting the ImageProcessingMethodConfiguration activity and processing its result are done in AndroidVision

private void handleConfigMethod()
{
	if(mMethod == null)
	{
		return;
	}

	IImageProcessingMethodConfiguration configuration = mMethod.getConfiguration();
	if(configuration == null)
	{
		Toast.makeText(this, "No configuration available", Toast.LENGTH_SHORT).show();;
		return;
	}
	Intent myIntent = new Intent(AndroidVision.this, ImageProcessingMethodConfiguration.class);			
	myIntent.putExtra(ImageProcessingMethodConfiguration.CONFIGURATION, (Parcelable)configuration);
	
	startActivityForResult(myIntent, REQUEST_METHODCONFIGURATION);
}

private void handleConfigMethodResult(int requestCode, int resultCode, Intent intent)
{
	IImageProcessingMethodConfiguration configuration = (IImageProcessingMethodConfiguration)intent.getExtras().getParcelable(ImageProcessingMethodConfiguration.CONFIGURATION);
	mMethod.setConfiguration(configuration);
	
	ApplyConfigurationOnObjectEditor();
}

private void ApplyConfigurationOnObjectEditor()
{
	//See above for the implementation
}

ImageProcessingMethodBase

All imageprocessing methods derive from ImageProcessingMethodBase and implement the IImageProcessingMethod interface:

public interface IImageProcessingMethod extends FrameGrabberViewBase.CvCameraViewDrawable {
	
	String getMethodId();
	
	boolean ToggleShowExtData();
	
	ImageType getInputImageType();
	ImageType getOutputImageType();
	
	Mat getInputImage();
	void setInputImage(Mat image);
	Mat getOutputImage();
	void setOutputImage(Mat image);
	
	IImageProcessingMethodConfiguration getConfiguration();
	void setConfiguration(IImageProcessingMethodConfiguration config);
	
	void Execute();
}
public abstract class ImageProcessingMethodBase implements IImageProcessingMethod {

	protected boolean showExtData = false;	
	
	public ImageProcessingMethodBase()
	{
		redPaint = new Paint();
		redPaint.setColor(Color.RED);
        redPaint.setTextSize(15);
		
        greenPaint = new Paint();
		greenPaint.setColor(Color.GREEN);
		greenPaint.setTextSize(15);
	}

	@Override
	public boolean ToggleShowExtData() {
		showExtData = !showExtData;
		return showExtData;
	}
	
	protected Paint redPaint;
	protected Paint greenPaint;
}

A sample is the BoxBlur class:

public class BoxBlur extends ImageProcessingMethodBase {

	public static final String METHOD_ID = "BOXBLUR";
	
	private Size sz;
	
	public BoxBlur()
	{
		configuration = new BoxConfiguration();
		configuration.setKernelSize(5);
		
		CopyConfiguration();
	}

	@Override
	public String getMethodId() {
		return METHOD_ID;
	}

	@Override
	public void setResolution(int width, int height) {
	}

	@Override
	public void draw(Canvas canvas) {
		if(!showExtData)
		{
			return;
		}
		
		CanvasTextPage page = new CanvasTextPage(canvas);
		page.startNewPage();
		page.drawTextLine(METHOD_ID, redPaint);
		page.drawTextLine("Kernel size: " + sz.width,  configuration.isPropertyAvailableInView("KernelSize") ? greenPaint : redPaint);

	}

	@Override
	public ImageType getInputImageType() {
		return ImageType.Gray;
	}

	@Override
	public ImageType getOutputImageType() {
		return ImageType.Gray;
	}

	@Override
	public Mat getInputImage() {
		return inputImage;
	}

	@Override
	public void setInputImage(Mat image) {
		inputImage = image;
	}

	@Override
	public Mat getOutputImage() {
		return outputImage;
	}

	@Override
	public void setOutputImage(Mat image) {
		outputImage = image;
	}

	@Override
	public IImageProcessingMethodConfiguration getConfiguration() {
		return configuration;
	}

	@Override
	public void setConfiguration(IImageProcessingMethodConfiguration config) {
		configuration = (BoxConfiguration)config;
	}

	@Override
	public void Execute() {
		CopyConfiguration(); 
    	Imgproc.blur(inputImage, outputImage, sz);
	}
	
	private void CopyConfiguration()
	{
		sz = new Size(configuration.getKernelSize(), configuration.getKernelSize()); 
	}

	BoxConfiguration configuration;
	Mat inputImage;
	Mat outputImage;
	
}

Applying the method happens in the AndroidVision.onCameraFrame of which the code is allready shown above.

Direct configuration editing in the cameraview

The Structure

The idea is to have a kind of objecteditor on top of the cameraview. This allows you to change the configuration of a method and immediately see the result.

In following text I will frequently use words like "control", "listview", etc... Don't let this fool you: they have nothing to do with the usual controls provided by Android. Instead, they are a bunch of classes mimicking some of the standard controls but providing custom drawing and touch handling. They all derive from the class CanvasWidgets

Based on these classes derived from CanvasWidgets are then a bunch of classes implementing property editors similar to my Object editor, allbeit much simpler.

Following is a simple scheme showing what is involved:


The code

CanvasWidgetBase

The base class implements some basic functionality, like positioning and setting the color.

public abstract class CanvasWidgetBase {

	public void setWidth(int width)
	{
		this.width = width;
		PositionChanged();
	}
	
	public int getWidth()
	{
		return this.width;
	}

	public void setHeight(int height)
	{
		this.height = height;
		PositionChanged();
	}
	
	public int getHeigth()
	{
		return this.height;
	}

	public void setX(int x)
	{
		this.x = x;
		PositionChanged();
	}
	
	public int getX()
	{
		return this.x;
	}

	public void setY(int y)
	{
		this.y = y;
		PositionChanged();
	}
	
	public int getY()
	{
		return this.y;
	}
	
	public int getColor() {
		return color;
	}

	public void setColor(int color) {
		this.color = color;
	}
	
	public void PositionChanged()
	{
		
	}
	
	public abstract void Draw(Canvas canvas);
    
    public abstract boolean onTouchEvent(MotionEvent motion); 

	protected int width;
	protected int height;
	protected int x;
	protected int y;
	
	protected int color = Color.BLUE;

	protected Paint paint = new Paint();
}

Derived from this are following classes:

  • ButtonCanvasWidget: implements a button
  • ItemsListCanvasWidget: implements an vertical list of CanvasWidgetBase controls
  • ListItemSelectorCanvasWidget: implements a control which allow horizontal scrolling through a list
  • SliderCanvasWidget: implements a slider

Following shows the code of the ListItemSelectorCanvasWidget:

public class ListItemSelectorCanvasWidget extends CanvasWidgetBase {

	public interface OnSelectHandler
	{
		void OnNext(ListItemSelectorCanvasWidget listItemSelectorCanvasWidget);
		void OnPrevious(ListItemSelectorCanvasWidget listItemSelectorCanvasWidget);
	}
	
	public ListItemSelectorCanvasWidget()
	{
		ChangeSelectedItemGesture clickGestureBuilder = new ChangeSelectedItemGesture(this);
		
		touchHandler = new TouchHandler();
		touchHandler.addGesture(clickGestureBuilder.create());
	}
	
	public void setOnSelectHandler(OnSelectHandler onSelectHandler)
	{
		this.onSelectHandler = onSelectHandler;
	}
	
	public OnSelectHandler getOnSelectHandler()
	{
		return this.onSelectHandler;
	}
	
	@Override
	public void Draw(Canvas canvas) {
		
		paint.setColor(getColor());
		paint.setStyle(Paint.Style.STROKE);		
		
		canvas.drawRect(getControlRectangle(), paint );
		
	}

	@Override
	public boolean onTouchEvent(MotionEvent motion) {
		
    	touchHandler.onTouchEvent(motion);
		
		return true;
	}
	
	public boolean HitTest(int xPos, int yPos)
	{
		Rect result = getControlRectangle();
		
		if(result.contains(xPos, yPos))
			return true;
		
		return false;
	}
	
	public boolean HitTest(ScreenVector point)
	{
		return HitTest(point.x, point.y);
	}
	
	private Rect getControlRectangle()
	{
		Rect result = new Rect(getX() + padding, 
				getY() + padding, 
				getX() + getWidth() - 2*padding, 
				getY() + getHeigth() - 2*padding);
		
		return result;
	}
	
	private OnSelectHandler onSelectHandler = null;
	private int padding = 5;
	private TouchHandler touchHandler;
}

As you can see, all touch handling is implemented using my Touch DSL library. These and the base class are the basis for a bunch of controls implementing the in view property editor.

PropertyProxyListInViewEditor

This class is the source for editing properties directly in the cameraview. It is a list of PropertyProxyInViewEditor which is the base class for a bunch of controls by which configuration properties can be edited in the cameraview.

public class PropertyProxyListInViewEditor extends ItemsListCanvasWidget {

	public void setPropertyList(ArrayList<PropertyProxy> propertyList)
	{
		this.propertyList = new ArrayList<PropertyProxy>(propertyList);
		
		ArrayList<PropertyProxyInViewEditor> editorList = new ArrayList<PropertyProxyInViewEditor>();
		for(PropertyProxy property : this.propertyList)
		{
			Class<?> editorType = getEditorType(property);
			PropertyProxyInViewEditor editor = instantiatePropertyProxyInViewEditorFromClass(editorType);
			editor.setPropertyProxy(property);
			editorList.add(editor);
		}
		this.setItemsList(editorList);
	}
	
	private Class<?> getEditorType(PropertyProxy property)
	{
		InViewEditorType editorType = property.getAnnotation(InViewEditorType.class);
		if(editorType == null)
			return null;
		return editorType.Type();
	}
	
	private static PropertyProxyInViewEditor instantiatePropertyProxyInViewEditorFromClass(Class<?> itemClass)
	{
		Constructor<?> cons;
		PropertyProxyInViewEditor canvasWidget = null;
		try {
			cons = itemClass.getConstructor();
			canvasWidget = (PropertyProxyInViewEditor)cons.newInstance();
		} catch (IllegalArgumentException e) {
			e.printStackTrace();
		} catch (InvocationTargetException e) {
			e.printStackTrace();
		} catch (SecurityException e) {
			e.printStackTrace();
		} catch (NoSuchMethodException e) {
			e.printStackTrace();
		} catch (InstantiationException e) {
			e.printStackTrace();
		} catch (IllegalAccessException e) {
			e.printStackTrace();
		}
		
		return canvasWidget;
	}
	
	private ArrayList<PropertyProxy> propertyList;

}
public abstract class PropertyProxyInViewEditor extends CanvasWidgetBase {
	private PropertyProxy propertyProxy;

	public void setPropertyProxy(PropertyProxy propPrxy) {
		propertyProxy = propPrxy;
	}

	public PropertyProxy getPropertyProxy() {
		return propertyProxy;
	}

}

Derived from this last control are:

  • AllowedValueSelectorInViewEditor: contains a ListItemSelectorCanvasWidget control allowing for the selection of items in a list by swiping inside the control.
  • IntegerSliderInViewEditor: contains a SliderCanvasWidget control allowing the set a value between a minimum and maximum by sliding, using a configurable step.
  • IntegerUpDownInViewEditor: contains two ButtonCanvasWidget controls allowing to set a value, optionally between a minimum and maximum value, using a configurable step.

As a sample you see the code of the IntegerUpDownInViewEditor class below:

public class IntegerUpDownInViewEditor extends PropertyProxyInViewEditor implements OnClickHandler {

	public IntegerUpDownInViewEditor()
	{
		leftButton.setOnClickHandler(this);
		leftButton.setColor(Color.argb(128, 0, 0, 255));
		rightButton.setOnClickHandler(this);
		rightButton.setColor(Color.argb(128, 0, 0, 255));
	}

	@Override
	public void setPropertyProxy(PropertyProxy propPrxy) {
		Class<?> validationType = propPrxy.getValidationType();
		if(validationType == IntegerRangeValidation.class)
		{
			IntegerRangeValidation validation = (IntegerRangeValidation)propPrxy.GetValidation();
			minValue = validation.MinValue();
			maxValue = validation.MaxValue();
		}
		
		IntegerStepValue stepValueAnnotation = propPrxy.getAnnotation(IntegerStepValue.class);
		if(stepValueAnnotation != null)
		{
			step = stepValueAnnotation.Step();
		}

		super.setPropertyProxy(propPrxy);
	}
	
	@Override
	public void setWidth(int width)
	{
		this.width = width;
		
		leftButton.setWidth(this.width/2);
		rightButton.setX(getX() + this.width/2);
		rightButton.setWidth(this.width/2);
	}
	
	@Override
	public void setHeight(int height)
	{
		this.height = height;
		leftButton.setHeight(this.height);
		rightButton.setHeight(this.height);
	}
	
	@Override
	public void setX(int x)
	{
		this.x = x;
		
		leftButton.setX(this.x);
		rightButton.setX(getX() + this.width/2);
	}
	
	@Override
	public void setY(int y)
	{
		this.y = y;
		
		leftButton.setY(this.y);
		rightButton.setY(this.y);
	}
	
	@Override
	public void OnClick(ButtonCanvasWidget buttonCanvasWidget) {
		int currentValue = (Integer)getPropertyProxy().getValue(int.class);
		int newValue = currentValue;
		if(buttonCanvasWidget == leftButton)
		{
			newValue = currentValue - step;
			if(newValue < minValue)
			{
				newValue = minValue;
			}
		}
		if(buttonCanvasWidget == rightButton)
		{
			newValue = currentValue + step;
			if(newValue > maxValue)
			{
				newValue = maxValue;
			}
		}
		getPropertyProxy().setValue(newValue);
	}
	
	@Override
	public void Draw(Canvas canvas) {
		
		paint.setColor(getColor());
		paint.setStyle(Paint.Style.STROKE);		
		
		leftButton.Draw(canvas);
		rightButton.Draw(canvas);
	}

	@Override
	public boolean onTouchEvent(MotionEvent motion) {
		leftButton.onTouchEvent(motion);
		rightButton.onTouchEvent(motion);
		return false;
	}

	private ButtonCanvasWidget leftButton = new ButtonCanvasWidget();
	private ButtonCanvasWidget rightButton = new ButtonCanvasWidget();
	
	private int minValue = Integer.MIN_VALUE;
	private int maxValue = Integer.MAX_VALUE;
	private int step = 1;
}

If a property can be edited in the cameraview and what editor to use are determined by the InViewEditable and InViewEditorType annotations respectively:

@Target({ ElementType.METHOD, ElementType.TYPE })
@Retention(RetentionPolicy.RUNTIME)
public @interface InViewEditable {
	boolean Editable() default true;
}
@Target({ ElementType.METHOD, ElementType.TYPE })
@Retention(RetentionPolicy.RUNTIME)
public @interface InViewEditorType {
	Class<?> Type();
	String ClassName() default "";
}

You can see their usage in the below sample code of the BoxConfiguration class

public class BoxConfiguration extends ImageProcessingMethodConfigBase implements Parcelable, IImageProcessingMethodConfiguration   {
	
	// More code
	
	@DataTypeViewer(Type=IntegerSliderViewer.class)
	public int getKernelSize()
	{
		return kernelSize;
	}
	
	@IntegerRangeValidation(MinValue = 1, MaxValue = 15, ErrorMessage = "Value must be between 1 and 15")
	@IntegerStepValue(Step = 2)
	@InViewEditable
	@InViewEditorType(Type=IntegerUpDownInViewEditor.class)
	public void setKernelSize(int value)
	{
		kernelSize = value;
		if(callback != null)
		{
			callback.PropertyChanged(this, "ANY", null, null);
		}
	}

	private OnPropertyChangedListener callback;
	@Override
	@HideSetter
	public void setPropertyChangedListener(OnPropertyChangedListener callback) {
		this.callback = callback;
	}
	
}

Setting up the in-view editor is done in the handleConfigMethodResult en ApplyConfigurationOnObjectEditor of the AndroidVision activity. See above for the code.

Supported imageprocessing methods

Overview

 

As mentioned above the imageprocessing methods are categorized. Currently following categories and their methods exist:

  • Blur
    • Box Blur
    • Gaussian Blur
    • Median Blur
  • Edge Detection
    • Canny Edge Detection
    • Sobel Edge Detection
  • Histogram
    • Histogram Equalization
  • Hough transform
    • Hough Circles
    • Hough Lines
  • Mathematical morphology
    • Dilation
    • Erosion

In the following section I will explain two methods: SobelEdgeDetection and HoughLines. I've chosen these two methods because the first is simple and the second is somewhat complex: as such you will see an example of both. I do invite you however to download the code and check the other methods.

The Code

SobelEdgeDetection

public class SobelEdgeDetection extends ImageProcessingMethodBase {

	public static final String METHOD_ID = "SOBEL";
	
	public SobelEdgeDetection()
	{
		configuration = new SobelConfiguration();
		configuration.setDerivXOrder(1);
		configuration.setDerivYOrder(1);
		configuration.setKernelSize(3);
		configuration.setScale(1.0);
		configuration.setDelta(0.0);
	}
	
	@Override
	public String getMethodId() {
		return METHOD_ID;
	}

	@Override
	public void setResolution(int width, int height) {
		// TODO Auto-generated method stub
		
	}

	@Override
	public void draw(Canvas canvas) {
		if(!showExtData)
		{
			return;
		}
		
		CanvasTextPage page = new CanvasTextPage(canvas);
		page.startNewPage();
		page.drawTextLine(METHOD_ID, redPaint);
		page.drawTextLine("dX order: " + configuration.getDerivXOrder(), configuration.isPropertyAvailableInView("DerivXOrder") ? greenPaint : redPaint);
		page.drawTextLine("dY order: " + configuration.getDerivYOrder(), configuration.isPropertyAvailableInView("DerivYOrder") ? greenPaint : redPaint);
		page.drawTextLine("Kernel size: " + configuration.getKernelSize(), configuration.isPropertyAvailableInView("KernelSize") ? greenPaint : redPaint);
		page.drawTextLine("Scale: " + configuration.getScale(), configuration.isPropertyAvailableInView("Scale") ? greenPaint : redPaint);
		page.drawTextLine("Delta: " + configuration.getDelta(), configuration.isPropertyAvailableInView("Delta") ? greenPaint : redPaint);

	}
	
	@Override
	public ImageType getInputImageType() {
		return ImageType.Color;
	}

	@Override
	public ImageType getOutputImageType() {
		return ImageType.Gray;
	}

	@Override
	public Mat getInputImage() {
		return inputImage;
	}

	@Override
	public void setInputImage(Mat image) {
		inputImage = image;
	}

	@Override
	public Mat getOutputImage() {
		return outputImage;
	}

	@Override
	public void setOutputImage(Mat image) {
		outputImage = image;
	}
	
	@Override
	public IImageProcessingMethodConfiguration getConfiguration()
	{
		return configuration;
	}
	
	@Override
	public void setConfiguration(IImageProcessingMethodConfiguration config)
	{
		configuration = (SobelConfiguration)config;
	}

	@Override
	public void Execute() {
		Imgproc.Sobel(inputImage, outputImage, 
				CvType.CV_8U, 
				configuration.getDerivXOrder(), 
				configuration.getDerivYOrder(), 
				configuration.getKernelSize(),
				configuration.getScale(), 
				configuration.getDelta());
	}

	SobelConfiguration configuration;
	Mat inputImage;
	Mat outputImage;
}

The SobelEdgeDetection is configured by SobelConfiguration

public class SobelConfiguration extends ImageProcessingMethodConfigBase implements Parcelable, IImageProcessingMethodConfiguration {
	int derivXOrder;
	int derivYOrder;
	int kernelSize; 
	double scale;
	double delta;
	
	public SobelConfiguration()
	{
		fillInViewEditingAllowed(this);
	}
	
	public SobelConfiguration(Parcel in) 
	{ 
		fillInViewEditingAllowed(this);
		readFromParcel(in);
	}
		
	@DataTypeViewer(Type=IntegerSliderViewer.class)
	public int getDerivXOrder()
	{
		return derivXOrder;
	}
	
	@IntegerRangeValidation(MinValue = 0, MaxValue = 2, ErrorMessage = "Value must be between 0 and 2")
	@InViewEditable
	@InViewEditorType(Type=IntegerUpDownInViewEditor.class)
	public void setDerivXOrder(int threshold)
	{
		derivXOrder = threshold;
		if(callback != null)
		{
			callback.PropertyChanged(this, "ANY", null, null);
		}
	}
		
	@DataTypeViewer(Type=IntegerSliderViewer.class)
	public int getDerivYOrder()
	{
		return derivYOrder;
	}
	
	@IntegerRangeValidation(MinValue = 0, MaxValue = 2, ErrorMessage = "Value must be between 0 and 2")
	@InViewEditable
	@InViewEditorType(Type=IntegerUpDownInViewEditor.class)
	public void setDerivYOrder(int threshold)
	{
		derivYOrder = threshold;
		if(callback != null)
		{
			callback.PropertyChanged(this, "ANY", null, null);
		}
	}
	
	@DataTypeViewer(Type=IntegerSliderViewer.class)
	public int getKernelSize()
	{
		return kernelSize;
	}
	
	@IntegerRangeValidation(MinValue = 3, MaxValue = 7, ErrorMessage = "Value must be between 3 and 7")
	@IntegerStepValue(Step = 2)
	@InViewEditable
	@InViewEditorType(Type=IntegerUpDownInViewEditor.class)
	public void setKernelSize(int value)
	{
		kernelSize = value;
		if(callback != null)
		{
			callback.PropertyChanged(this, "ANY", null, null);
		}
	}
	
	public double getScale()
	{
		return scale;
	}
	
	public void setScale(double value)
	{
		scale = value;
	}
	
	public double getDelta()
	{
		return delta;
	}
	
	public void setDelta(double value)
	{
		delta = value;
	}

	private OnPropertyChangedListener callback;
	@Override
	@HideSetter
	public void setPropertyChangedListener(OnPropertyChangedListener callback) {
		this.callback = callback;
	}

	@Override
	public int describeContents() {
		return 0;
	}
	
	public void readFromParcel(Parcel in) {
		baseReadFromParcel(in);
		derivXOrder = in.readInt();
		derivYOrder = in.readInt(); 
		kernelSize = in.readInt();
		scale = in.readDouble();
		delta = in.readDouble();
	}

	@Override
	public void writeToParcel(Parcel arg0, int arg1) {
		baseWriteToParcel(arg0, arg1);
		arg0.writeInt(derivXOrder);
		arg0.writeInt(derivYOrder);
		arg0.writeInt(kernelSize);
		arg0.writeDouble(scale);
		arg0.writeDouble(delta);
	}
	
	public static final Parcelable.Creator<SobelConfiguration> CREATOR = new Parcelable.Creator<SobelConfiguration>()
	{
		@Override
		public SobelConfiguration createFromParcel(Parcel source) {
			return new SobelConfiguration(source);
		}

		@Override
		public SobelConfiguration[] newArray(int size) {
			return new SobelConfiguration[size];
		}
	};
}

HoughLines

public class HoughLines extends ImageProcessingMethodBase {

	public static final String METHOD_ID = "HOUGHLINES";
	
	public HoughLines()
	{
		configuration = new HoughLinesConfiguration();
		configuration.setLowerThreshold(125);
		configuration.setUpperThreshold(350);
		configuration.setApertureSize(3);
		configuration.setL2Gradient(false);
		
		configuration.setRho(1);
		configuration.setTheta(1);
		configuration.setThreshold(80);
		
		CopyConfiguration();
	}
	
	@Override
	public String getMethodId() {
		return METHOD_ID;
	}

	@Override
	public void setResolution(int width, int height) {
		// TODO Auto-generated method stub
		
	}

	@Override
	public void draw(Canvas canvas) {
		if(!showExtData)
		{
			return;
		}
		
		CanvasTextPage page = new CanvasTextPage(canvas);
		page.startNewPage();
		page.drawTextLine(METHOD_ID, redPaint);

		page.drawTextLine("Lower threshold: " + lowerThreshold, configuration.isPropertyAvailableInView("LowerThreshold") ? greenPaint : redPaint);
		page.drawTextLine("Upper threshold: " + upperThreshold, configuration.isPropertyAvailableInView("UpperThreshold") ? greenPaint : redPaint);
		page.drawTextLine("Aperture size: " + apertureSize, configuration.isPropertyAvailableInView("ApertureSize") ? greenPaint : redPaint);
		page.drawTextLine("L2 gradient: " + L2Gradient, configuration.isPropertyAvailableInView("L2Gradient") ? greenPaint : redPaint);

		page.drawTextLine("Rho: " + rho, configuration.isPropertyAvailableInView("Rho") ? greenPaint : redPaint);
		page.drawTextLine("Theta: " + theta, configuration.isPropertyAvailableInView("Theta") ? greenPaint : redPaint);
		page.drawTextLine("Threshold: " + threshold, configuration.isPropertyAvailableInView("Threshold") ? greenPaint : redPaint);
	}
	
	@Override
	public ImageType getInputImageType() {
		return ImageType.Color;
	}

	@Override
	public ImageType getOutputImageType() {
		return ImageType.Gray;
	}

	@Override
	public Mat getInputImage() {
		return inputImage;
	}

	@Override
	public void setInputImage(Mat image) {
		inputImage = image;
	}

	@Override
	public Mat getOutputImage() {
		return outputImage;
	}

	@Override
	public void setOutputImage(Mat image) {
		outputImage = image;
	}
	
	@Override
	public IImageProcessingMethodConfiguration getConfiguration()
	{
		return configuration;
	}
	
	@Override
	public void setConfiguration(IImageProcessingMethodConfiguration config)
	{
		configuration = (HoughLinesConfiguration)config;
		
		CopyConfiguration();
	}

	@Override
	public void Execute() {

		double thetaInRadians = theta * 180 / Math.PI;
		
		Mat cannyMat = new Mat();
    	Mat lineMat = new Mat();
		Imgproc.Canny(inputImage, outputImage, lowerThreshold, upperThreshold, apertureSize, L2Gradient);
		Imgproc.Canny(inputImage, cannyMat, lowerThreshold, upperThreshold, apertureSize, L2Gradient);
		Imgproc.HoughLines(cannyMat, lineMat, rho, thetaInRadians, threshold);
		
		DrawHoughLines(lineMat);
	}
	
	private void DrawHoughLines(Mat lineMat)
	{
		Scalar color = new Scalar(255, 255, 255); 
		
		double[] data; 
		double rho, theta; 
		Point pt1 = new Point(); 
		Point pt2 = new Point(); 
		double a, b; 
		double x0, y0; 
		for (int i = 0; i < lineMat.cols(); i++) {     
			data = lineMat.get(0, i);
			rho = data[0];     
			theta = data[1];     
			a = Math.cos(theta);     
			b = Math.sin(theta);     
			x0 = a*rho;     
			y0 = b*rho;     
			pt1.x = Math.round(x0 + 1000*(-b));     
			pt1.y = Math.round(y0 + 1000*a);     
			pt2.x = Math.round(x0 - 1000*(-b));     
			pt2.y = Math.round(y0 - 1000 *a);     
			Core.line(outputImage, pt1, pt2, color, 3); 
		} 		
	}
	
	private void CopyConfiguration()
	{
		lowerThreshold = configuration.getLowerThreshold(); 
		upperThreshold = configuration.getUpperThreshold(); 
		apertureSize = configuration.getApertureSize();
		L2Gradient = configuration.getL2Gradient();
		
		rho = configuration.getRho();
		theta = configuration.getTheta();
		threshold = configuration.getThreshold();
	}

	HoughLinesConfiguration configuration;
	Mat inputImage;
	Mat outputImage;
	
	int lowerThreshold;
	int upperThreshold;
	int apertureSize;
	boolean L2Gradient;
	
	double rho;
	double theta;
	int threshold;

}

The HoughLines is configured by HoughLinesConfiguration

public class HoughLinesConfiguration extends ImageProcessingMethodConfigBase implements Parcelable, IImageProcessingMethodConfiguration {
	int lowerThreshold; 
	int upperThreshold; 
	int apertureSize;
	boolean L2Gradient;
	
	int rho;
	int theta;
	int threshold;
	
	public HoughLinesConfiguration()
	{
	
	}
	
	public HoughLinesConfiguration(Parcel in) 
	{ 
		lowerThreshold = in.readInt();
		upperThreshold = in.readInt(); 
		apertureSize = in.readInt();
		L2Gradient = in.readInt()==1?true:false;
		
		rho = in.readInt();
		theta = in.readInt();
		threshold = in.readInt();
	}
		
	@Category(Name="Canny")
	public int getLowerThreshold()
	{
		return lowerThreshold;
	}
	
	@InViewEditable
	@InViewEditorType(Type=IntegerSliderInViewEditor.class)
	@IntegerRangeValidation(MinValue = 0, MaxValue = 200, ErrorMessage = "Value must be between 0 and 200")
	public void setLowerThreshold(int threshold)
	{
		lowerThreshold = threshold;
		if(callback != null)
		{
			callback.PropertyChanged(this, "ANY", null, null);
		}
	}
	
	@Category(Name="Canny")
	public int getUpperThreshold()
	{
		return upperThreshold;
	}
	
	@InViewEditable
	@InViewEditorType(Type=IntegerSliderInViewEditor.class)
	@IntegerRangeValidation(MinValue = 0, MaxValue = 200, ErrorMessage = "Value must be between 0 and 200")
	public void setUpperThreshold(int threshold)
	{
		upperThreshold = threshold;
		if(callback != null)
		{
			callback.PropertyChanged(this, "ANY", null, null);
		}
	}

	@Category(Name="Canny")
	public int getApertureSize() {
		return apertureSize;
	}

	@InViewEditable
	@InViewEditorType(Type=IntegerUpDownInViewEditor.class)
	public void setApertureSize(int value) {
		apertureSize = value;
		if(callback != null)
		{
			callback.PropertyChanged(this, "ANY", null, null);
		}
	}

	@Category(Name="Canny")
	public boolean getL2Gradient() {
		return L2Gradient;
	}

	public void setL2Gradient(boolean value) {
		L2Gradient = value;
		if(callback != null)
		{
			callback.PropertyChanged(this, "ANY", null, null);
		}
	}

	@Category(Name="Hough transform")	
	public int getRho() {
		return rho;
	}

	public void setRho(int value) {
		rho = value;
		if(callback != null)
		{
			callback.PropertyChanged(this, "ANY", null, null);
		}
	}

	@Category(Name="Hough transform")	
	public int getTheta() {
		return theta;
	}

	@IntegerRangeValidation(MinValue = 1, MaxValue = 360, ErrorMessage = "Value must be between 1 and 360")
	@InViewEditable
	@InViewEditorType(Type=IntegerUpDownInViewEditor.class)
	public void setTheta(int value) {
		theta = value;
		if(callback != null)
		{
			callback.PropertyChanged(this, "ANY", null, null);
		}
	}

	@Category(Name="Hough transform")	
	public int getThreshold() {
		return threshold;
	}

	@InViewEditable
	@InViewEditorType(Type=IntegerUpDownInViewEditor.class)
	public void setThreshold(int value) {
		threshold = value;
		if(callback != null)
		{
			callback.PropertyChanged(this, "ANY", null, null);
		}
	}

	private OnPropertyChangedListener callback;
	@Override
	@HideSetter
	public void setPropertyChangedListener(OnPropertyChangedListener callback) {
		this.callback = callback;
	}

	@Override
	public int describeContents() {
		return 0;
	}

	@Override
	public void writeToParcel(Parcel arg0, int arg1) {
		arg0.writeInt(lowerThreshold);
		arg0.writeInt(upperThreshold);
		arg0.writeInt(apertureSize);
		arg0.writeInt(L2Gradient?1:0);
		
		arg0.writeInt(rho);
		arg0.writeInt(theta);
		arg0.writeInt(threshold);
	}
	
	public static final Parcelable.Creator<HoughLinesConfiguration> CREATOR = new Parcelable.Creator<HoughLinesConfiguration>()
	{
		@Override
		public HoughLinesConfiguration createFromParcel(Parcel source) {
			return new HoughLinesConfiguration(source);
		}

		@Override
		public HoughLinesConfiguration[] newArray(int size) {
			return new HoughLinesConfiguration[size];
		}
	};
}

Conclusion

Well, this is about it for this release. I hope you enjoyed the article. As mentioned, this is the "get it out now" release so there is (a lot of) room for improvement like more methods, nicer interface, etc. And thus...

What is next?

There are some more features on my wish list:

  • Implement more methods
  • Workflow functionality
  • Multithreading support
  • Implement other image sources like the front facing camera, the image galery
  • Make the interface nicer (animations, nicer controls, ...)

Version history

  • Version 1.0: Initial version
  • Version 1.1: Code cleanup

License

This article, along with any associated source code and files, is licensed under The Code Project Open License (CPOL)

Share

About the Author

Serge Desmedt
Software Developer (Senior)
Belgium Belgium
No Biography provided
Follow on   LinkedIn

Comments and Discussions

 
GeneralMy vote of 5 PinprofessionalMihai MOGA14-Feb-14 17:08 
GeneralRe: My vote of 5 PinmemberSerge Desmedt14-Feb-14 21:33 
QuestionEclipse Kepler Android Build Issues PinmemberJ Forde9-Jan-14 7:18 
AnswerRe: Eclipse Kepler Android Build Issues PinmemberSerge Desmedt10-Jan-14 8:23 
GeneralRe: Eclipse Kepler Android Build Issues PinmemberJ Forde12-Jan-14 7:49 
AnswerRe: Eclipse Kepler Android Build Issues PinmemberSerge Desmedt14-Jan-14 7:37 
GeneralRe: Eclipse Kepler Android Build Issues PinmemberJ Forde16-Jan-14 3:36 

General General    News News    Suggestion Suggestion    Question Question    Bug Bug    Answer Answer    Joke Joke    Rant Rant    Admin Admin   

Use Ctrl+Left/Right to switch messages, Ctrl+Up/Down to switch threads, Ctrl+Shift+Left/Right to switch pages.

| Advertise | Privacy | Mobile
Web02 | 2.8.140916.1 | Last Updated 20 Jan 2014
Article Copyright 2014 by Serge Desmedt
Everything else Copyright © CodeProject, 1999-2014
Terms of Service
Layout: fixed | fluid