WP 8.1 is one of major update for Windows Phone. WP 8.1 has unveiled new APIs, new universal apps concept, inclusion of WinRT APIs, new UI controls and much more. One of new API introduced is media editing API.
In earlier days, we were using Movie Maker or other 3rd party desktop softwares to trim the video, to make a composition of multiple video, to apply various effects etc. We were required to have a computer with more memory & high-end processors, but now our smartphones are enough powerful to do such tasks. So let’s get a deep dive in new media editing API of Windows Phone 8.1.
Windows Media Editing API
Media editing API has MediaClip class, whose object represents a single media i.e. video, image or a solid color video clip. To perform any kind of editing operation on a media file, you must first create MediaClip object. MediaClip provides three static method to create object.
Which creates a MediaClip object from a video file.
MediaClip.CreateFromImageFileAsync(IStorageFile file, TimeSpan originalDuration)
Which creates a MediaClip object from an image file. The parameter “originalDuration” has use while we create composition of media files. It represents how long the image would be displayed in the composed video clip.
MediaClip.CreateFromColor(Color color, TimeSpan originalDuration)
Which creates a solid color video clip that displays a single color for a specified length of time. Solid color video clips are typically used to create an explicit gap between video segments.
After creating media clip, you will have object of particular file. Then you can apply effects as well as editing. Let’s go through one by one.
Trimming the video is now just two steps process. MediaClip object has two properties (i) TrimTimeFromStart (ii) TrimTimeFromEnd. After creating MediaClip, you just have to set these two properties. After setting it, the MediaClip will have trimmed media. You can change later on the trimming times as many times as you want. We will see further how to render this MediaClip into a file.
MediaClip objMediaClip = await MediaClip.CreateFromFileAsync(file)
objMediaClip.TrimTimeFromStart = new TimeSpan(0,1,20);
objMediaClip.TrimTimeFromEnd = objMediaClip.OriginalDuration - new TimeSpan(0,2,9);
Slow motion & video stabilization effects
Microsoft has MFT (Media Foundation Transform) platform, through which one can developer C++ WinRT components. These can be used to apply various video playback & filter effects.
Windows.Media.Effects namespace is dedicated to pre-defined effects, which can be used to apply on any video file. One of effect is slow motion effects. Windows.Media.Effects has class SlowMotionEffectDefinition, which is used to apply slow motion effects to media. That class has one property called TimeStretchRate. It refers how slower the video will be. It takes double value of less than 1. Hence, if you set TimeStretchRate as 0.5, the video speed will be half of original. Setting its value greater than 1 throws exception. So don’t consider its use for fast motion video effects.
Now to apply the slow motion effect to any video, first you have to create its MediaClip object which is essential for media editing. MediaClip object has IList<IVideoEffectDefinition> property called VideoEffectDefinitions. So first all you have to create object of SlowMotionEffectDefinition and set TimeStretchRate property. Then add that object to VideoEffectDefinitions list. That’s it, your video is now in slow motion. We will see later on how to preview and how to save video with effects.
var objSlowMotionEffectDefinition = new SlowMotionEffectDefinition();
objSlowMotionEffectDefinition.TimeStretchRate = 0.6;
Similarly you can also add video stabilization effect. The VideoStabilization effect can help reduce shakiness in video. That effect resides in Windows.Media namespace. It has only class named VideoEffects, which is static class. To add this effect you don’t have to create an object. It has static string property VideoStabilization, which is activatable class ID of that video effect definition.
var objVideoStabilization = new VideoEffectDefinition(VideoEffects .VideoStabilization);
Filter & other video effects
Nokia Imaging SDK provides a plethora of filters for image processing. Unfortunately, it doesn’t work for videos. We can solve this problem by developing C++ WinRT components using MFT. According to MSDN, Media Foundation transforms (MFTs) provide a generic model for processing media data. MFTs are used for decoders, encoders, and digital signal processors (DSPs). In short, anything that sits in the media pipleine between the media source and the media sink is an MFT.
MSDN has provided a sample with several effect source code, wiz grayscale, fisheye, pinch, warp and invert. The sample also shows how to apply these effects in playback. No words for saving after applying the effects. So to apply MSDN sample’s effect is quite similar to slow motion & video stabilization effect. You have to first add WinRT C++ components from MSDN sample to your own project and then you have to define activatable class IDs in WP 8.1 project’s manifest file. Add below given lines by opening manifest file in text editor before </Package>.
<ActivatableClass ActivatableClassId="GrayscaleTransform.GrayscaleEffect" ThreadingModel="both"/>
<ActivatableClass ActivatableClassId="PolarTransform.PolarEffect" ThreadingModel="both"/>
<ActivatableClass ActivatableClassId="InvertTransform.InvertEffect" ThreadingModel="both"/>
Now each effect has its own activatable class ID, you have to first create object of VideoEffectDefinition class & pass that activatable class ID as argument in constructor. Finally add object of VideoEffectDefinition to MediaClip, that’s it.
var objGrayScaleEffect = new VideoEffectDefinition("GrayscaleTransform.GrayscaleEffect");
var objInvertEffect = new VideoEffectDefinition("InvertTransform.InvertEffect");
In MSDN sample fisheye, pinch & warp effect is defined in single WinRT componenet. To apply any of effect among them you need to pass configuration parameter, with activatable class. VideoEffectDefinition's constructor has one more overload, which takes a key-value pair to configure the effect.
PropertySet configuration = new PropertySet();
var objPolarEffect = new VideoEffectDefinition("PolarTransform.PolarEffect", configuration);
We have learned how to apply effects & trimming, now it’s turn to save the video. MediaEditing API has one more important class called MediaComposition. MediaClip represents a single media file, while MediaComposition is a group of MediaClip objects. It allows you manage all the MediaClip objects in composition.
Composing a video with multiple clips is complex task, it is better to have preview of it, isn’t it? MediaComposition provides a preview of itself. So to preview the video on which you have applied effects, you have to first add MediaClip object to MediaComposition and then generate preview stream. Set that stream as source to MediaElement. You can also set height & width of preview.
var objMediaComposition = new MediaComposition();
MediaElement.SetSource(objMediaComposition. GeneratePreviewMediaStreamSource(360, 240));
Now to save MediaClip you have to call its asynchronous method RenderToFileAsync(IStorageItem file).
var file = await ApplicationData.Current.LocalFolder.CreateFileAsync(“MyVideo.mp4”);
That’s it, your dream of media editing on Windows Phone is accomplished. Now you can edit media files from your WP apps.