This is a proposal to design an API and an application uses a camera to detect one hand gesture of user to control functionality of the application, e.g. a media player. The camera (the front camera of a tablet, a web cam or similar input device of a tablet PC or AIO PC) is used to detect one hand gesture of the user by applying image recognition methods.
Title: Gestures controlled media player.
- This app shall use the front camera of a tablet, web cam or similar input device of a tablet PC or AIO PC to detect one hand gesture of the user (at the beginning). The hand gesture will be detected in certain limited distance (not so near and not so far) form camera. The gesture could be, pointing with the hand to right, to left up and down, stop sign with the hand and finally a fist
- It shall control the main functions of a media player using the detected gesture like, start, stop, next, previous, pause, etc…
- API for gesture detection using a camera
- Service that can be used by all application which want to use gesture detection
- A Media player that use the API/service. And be controlled by user gesture
- Gesture is limited to a certain distance to camera, thus it could be possible to control several Apps
- Doing gesture in different plains parallel to the camera Z axis.
- In the next version of the APP and after evaluation of the capabilities, it might be possible to recognize more complicated gesture
- Windows 8 based AOI platform and
- Laptop windows 7 professional as development platform
- Microsoft visual studio,
- External SDK,
- Mathematic algorithm libs,
- Image signal processing Libs,
- Windows SDK
Language and design steps (simplified):
C++ to develop the API
The application will be also in C++
1. Will write the gesture detection methods.
2. Write a simplified Windows Application to debug detection
3. Implement a simplified media player e.g. For audio (or perhaps also for video)
4. Documentation, tutorial, video demo.
Points of Interest
Development of this application and corresponding API allows windows based application to react on gesture. At the beginning only one few hand gesture shall be detected, but the development will continue and shall be the enabler for further more complicated gesture based control of the applications.
Doing this the designer and with that Intel will gather experiences in development of image recognition algorithm and close the gap between reality and machine world.
It allows to detect the strengths and weaknesses of HW like sensors, camera, acceleration and performance of the platform e.g. by doing real time signal processing tasks, thus give hints for different areas of enhancements
Enable more Augment Reality developments.
H_01: 2013.09.03: Author, Proposal formulated for this work.
I am working in electronic industry for more than 25 year, 9 years of video signal processing and rest in telecommunication signal processing for 3G and 4G. My current position is system engineering platform system. I have experience in developing embedded SW design, DSP design, Controller SW, FPGA design, HW/SW codesign, java based application and application suites and many years of system engineering. Last application that I develop was an Android App using face/head detection algorithm with the mobile front camera to control content of the mobile display. The content was adapted to the position of the user watching the mobile display.