Animating Interactive 2D Elements in a 3D Panel

• Download the source code (requires Visual Studio 2008) - 4.02 MB
• Download the demo app (requires .NET Framework 3.5) - 19.81 KB

Introduction

This article reviews a custom WPF panel, named Panel3D, which hosts two-dimensional elements in three-dimensional space. You can supply an instance of Panel3D with objects to display, or you can use it as the items host of an ItemsControl. If you use Panel3D as the items host of a ListBox, the front-most item is automatically the selected item in the ListBox. Panel3D arranges its 3D models along a straight path, and provides animated movement of 3D models along that path. In addition, the panel implements a simple form of UI virtualization so that it runs quickly even if it contains hundreds of items.

Background

WPF provides excellent support for creating 2D and 3D user interfaces. As WPF matures, these two realms of programming are mixing, allowing 3D scenes to display interactive 2D elements. Despite the progress seen thus far, some seemingly basic concepts are still not supported out-of-the-box by WPF. One of those artificial limitations is that there is no built-in support for hosting a panel's child elements in 3D space. This article shows how to get around that limitation, by using my Panel3D class.

Brief History of Panel3D

I certainly cannot claim to be the sole inventor of this custom 3D panel. Not too long ago, my friend and fellow WPF Disciple, Sacha Barber, asked me to review a WPF project of his. He was working on a way to create a custom panel that painted its child elements in a Viewport3D, by painting the 3D models with a VisualBrush that referenced the 2D element. That demo project got me thinking about a similar problem: how to host the panel's 2D elements in 3D space. I wanted to figure out a way to put the panel's children into 3D space, not just paint a 3D model with a VisualBrush. I also wanted this panel to work as the items host of an ItemsControl.

After a couple trials and tribulations, I figured out a hacky way to do it. I cloned the panel's child elements and hosted the clones in a Viewport3D. It was definitely not ideal, but it worked. Fortunately, this caught another WPF Disciple's attention: Dr. WPF. The good Doctor put his brilliant mind to work and ended up figuring out a way to create a panel whose child elements can be hosted outside of the panel itself. My Panel3D derives from his LogicalPanel, so if you are interested in understanding the bigger picture, I recommend you read his superb article about it here.

Introducing Panel3D

Panel3D has a simple API. It exposes a few public dependency properties:

• AllowTransparency — Gets/sets whether the models in the scene support being truly translucent, such that the models behind them are visible through the models in front. The default value is false.
• AutoAdjustOpacity — Gets/sets whether the Panel3D automatically adjusts each model's opacity based on its visual index. The default value is true.
• Camera — Gets/sets a camera used to view the 3D scene.
• IsMovingItems — Returns whether or not the models displayed in this Panel3D are currently animating to new locations as a result of calling the MoveItems method.
• DefaultAnimationLength — Gets/sets the amount of time it takes to move items. This value can be overridden when calling MoveItems. The default value is 700 milliseconds.
• ItemLayoutDirection — Gets/sets a Vector3D that describes the direction in which the items are positioned. The default value is (-1, +1.3, -7).
• MaxVisibleModels — Gets/sets the maximum number of 3D models that can be displayed at once. The default value is 10. The minimum value for this property is 2.

Panel3D declares two public methods, one of which has two overloads:

• int GetVisibleIndexFromChildIndex(int childIndex) — Returns the visible index of the 3D model that represents the 2D element at the specified index in the panel's Children collection. Both index values are zero-based. The visible index of the front model is 0, and each successive model in the 3D scene has a visible index one higher than the previous model. If the element at the specified index is not currently in the viewport, the visible index is -1. This method is useful when you want to bring a model to the front of the 3D scene after the user clicks on it.
• void MoveItems(int itemCount, bool forward) — Moves the items forward or backward over the default animation length.
• void MoveItems(int itemCount, bool forward, TimeSpan animationLength) — Moves the items forward or backward over the specified animation length.

The class also exposes a bubbling routed event named ItemsHostLoaded, which is useful when the panel is the items host of an ItemsControl. Adding a handler for this event enables you to get a reference to the Panel3D, because, for whatever reason, its Loaded event does not bubble up in that situation.

Using Panel3D

You can easily create a Panel3D and give it some children to display. Here is the XAML found in the DirectWindow.xaml file, which is part of the demo project.

<pnl3D:Panel3D xmlns:pnl3D="clr-namespace:Panel3DLib;assembly=Panel3DLib">
  <TextBox
    AcceptsReturn="True"
    MaxLines="8"
    Text="Howdy"
    Width="100" Height="100"
    />

  <Button Width="100" Height="100">Destroy Universe</Button>

  <CheckBox IsChecked="True">Is this cool?</CheckBox>
</pnl3D:Panel3D>

That XAML looks exactly like the XAML seen for adding elements to any other WPF panel. Since Panel3D indirectly derives from the abstract Panel class, you can use it just like any other panel. After running the program and editing the TextBox, the UI looks like this:

If you decide that you would like to use Panel3D as the items host of an ItemsControl, you can use the XAML seen below:

<ItemsControl
  xmlns:pnl3D="clr-namespace:Panel3DLib;assembly=Panel3DLib"
  ItemsSource="{Binding Path=FooCollection}"
  >
  <ItemsControl.ItemsPanel>
    <ItemsPanelTemplate>
      <pnl3D:Panel3D />
    </ItemsPanelTemplate>
  </ItemsControl.ItemsPanel>
</ItemsControl>

The snippet seen above uses the standard ItemsPanel property of ItemsControl to specify that a Panel3D will host the control's items. You could also use Panel3D as the items host for a ListBox, which provides the notion of item selection. The selected item in a Panel3D is always the item in front. Panel3D does not support displaying multiple selected items at once.

You can make use of some more advanced features of Panel3D by simply setting some properties in XAML. Here is one panel configuration provided in the demo project's Panel3DConfigurations.xaml file:

<ItemsPanelTemplate x:Key="Across.Right">
  <pnl3D:Panel3D
    DefaultAnimationLength="0:0:0.5"
    ItemLayoutDirection="3.8, .55, -6.831"
    >
    <pnl3D:Panel3D.Camera>
      <PerspectiveCamera
        LookDirection="3, 0.8, -10"
        Position="+1.75, 0, 5"
        UpDirection="0, 1, 0"
        />
    </pnl3D:Panel3D.Camera>
  </pnl3D:Panel3D>
</ItemsPanelTemplate>

When you apply those settings and view BoundWindow in the demo app, the UI looks like this:

As seen above, adjusting the Panel3D's ItemLayoutDirection and Camera properties can drastically alter the way we view the items it hosts. Panel3D arranges its items in 3D space along a straight line defined by the Vector3D returned by the ItemLayoutDirection property. The camera object returned by the Camera property determines the angle from which we view that line of 3D models. The demo project contains a simple run-time editor for the ItemLayoutDirection property, as seen in the annotated screenshot below:

Putting 2D Elements into 3D Space

One of the fundamental tasks Panel3D must perform is mapping 2D elements to 3D models. When a child element is added to the panel, it must create a 3D model that hosts the 2D child element and display it at the correct location. When that 2D child element is removed, the corresponding 3D model must also be removed.

Here is the primary method responsible for this aspect of Panel3D:

protected override void OnLogicalChildrenChanged(
  UIElement elementAdded, UIElement elementRemoved)
{
    // Do not create a model for the Viewport3D.
    if (elementAdded == _viewport)
        return;

    bool add =
        elementAdded != null &&
        !_elementTo3DModelMap.ContainsKey(elementAdded);

    if (add)
        this.AddModelForElement(elementAdded);

    bool remove =
        elementRemoved != null &&
        _elementTo3DModelMap.ContainsKey(elementRemoved);

    if (remove)
        this.RemoveModelForElement(elementRemoved);
}

That method overrides a method inherited from Dr. WPF's LogicalPanel. In response to receiving a new logical child, this method executes:

void AddModelForElement(UIElement element)
{
    var model = BuildModel(element);

    // Add the new model at the correct location in our list of models.
    int idx = base.Children.IndexOf(element);
    _models.Insert(idx, model);

    _elementTo3DModelMap.Add(element, model);

    // If the scene has more than just a light source, grab the first
    // element and use it as the front model.  Otherwise, the scene
    // does not have any of our models in it yet, so pass the new one.
    var frontModel =
        _viewport.ModelCount > 0 ?
        _viewport.FrontModel :
        model;

    this.BuildScene(frontModel);
}

We will not bother looking at what happens when a logical child is removed, since it is essentially the opposite of the method seen above. The method that actually creates a new 3D model is invoked at the top of the preceding method. Now let us turn our attention to that logic:

/// <summary>
/// Returns an interactive 3D model that hosts
/// the specified UIElement.
/// </summary>
Viewport2DVisual3D BuildModel(UIElement element)
{
    var model = new Viewport2DVisual3D
    {
        Geometry = new MeshGeometry3D
        {
            TriangleIndices = new Int32Collection(
                new int[] { 0, 1, 2, 2, 3, 0 }),
            TextureCoordinates = new PointCollection(
                new Point[]
                    {
                        new Point(0, 1),
                        new Point(1, 1),
                        new Point(1, 0),
                        new Point(0, 0)
                    }),
            Positions = new Point3DCollection(
                new Point3D[]
                    {
                        new Point3D(-1, -1, 0),
                        new Point3D(+1, -1, 0),
                        new Point3D(+1, +1, 0),
                        new Point3D(-1, +1, 0)
                    })
        },
        Material = new DiffuseMaterial(),
        Transform = new TranslateTransform3D(),
        // Host the element in the 3D object.
        Visual = element
    };

    Viewport2DVisual3D.SetIsVisualHostMaterial(model.Material, true);

    return model;
}

That method creates and configures a Viewport2DVisual3D object that hosts the panel's 2D child element. It establishes that relationship by setting the Visual property to the 2D element, and then setting the attached IsVisualHostMaterial property to true on its Material. Learn more about Viewport2DVisual3D here.

Virtualization of the 3D Scene

UI virtualization is a standard technique for improving performance when dealing with hundreds or thousands of items in a list. The idea is that you only create and keep around UI elements for the items that are currently in view. When you bring an item into view, UI elements are created for it. When scrolled out of view, the UI elements for that item are thrown away. This can have a huge impact on the memory footprint required to display a large number of items, since usually only a small fraction of them are in view at any given moment.

Panel3D employs a different type of UI virtualization. While developing the panel, I found that when it contains many items, calling the MoveItems method on it resulted in a very slow animation. The problem was that all of the 3D models were in the viewport, and each needed to animate to its new location and opacity. That takes a lot of processing power.

I decided to limit the number of items that the viewport can display to ten, so that item movement is much faster. When one model moves out of view, I add another to the viewport. This gives the illusion that all of the items are in the viewport.

It is important to note, however, that I do not lazily create or destroy the 3D models; I simply add and remove them to/from the viewport as necessary. If you want to display ten thousand items, the memory footprint of all those 3D models may be prohibitively expensive. In that case, you will need to take on the much more challenging task of implementing a VirtualizingPanel3D that does true UI virtualization. Have fun with that!

My simplistic UI virtualization scheme exists in two places. The BuildScene method, invoked by the AddModelForElement method seen in the previous section, limits the number of models added to the scene. That method is below:

/// <summary>
/// Tears down the current 3D scene and constructs a new one
/// where the specified model is the front object in view.
/// </summary>
void BuildScene(Viewport2DVisual3D frontModel)
{
    _viewport.RemoveAllModels();

    // Add in some 3D models, starting with the one in front.
    var current = frontModel;
    for (int i = 0; _viewport.ModelCount < this.MaxVisibleModels; ++i)
    {
        this.ConfigureModel(current, i);

        _viewport.AddToBack(current);

        current = this.GetNextModel(current);
        if (_viewport.Children.Contains(current))
            break;
    }
}

The other piece of the virtualization puzzle is in the code that moves items in the scene. We examine that logic next.

Animating 3D Models along a Path

The most challenging aspect of developing Panel3D, by far, was the item movement algorithm. This took me a while to get right. That logic must consider many things, including the UI virtualization concerns discussed in the prior section.

The way it works is actually quite simple. If the items are moved one position, the front or back item moves to the opposite end of the Viewport3D's Children collection. If the new front item is a ListBoxItem, I set the owning ListBox's SelectedItem property to the item. The real magic happens next, when we fire off a bunch of animations to move the items and, optionally, adjust their opacity. That logic uses the ordinal index of a model in the viewport’s Children collection to determine where it should exist, and how opaque it should be. Once all of the animations are in action, a DispatcherTimer is started. When the timer ticks some clean-up code executes. That clean-up code enforces the rule that only a certain number of models can exist in the viewport at once (as determined by the MaxVisibleModels property).

The main method of this algorithm is below:

/// <summary>
/// Moves the items forward or backward over the specified animation length.
/// </summary>
public void MoveItems(int itemCount, bool forward, TimeSpan animationLength)
{
    bool go = this.MoveItems_CanExecute(itemCount, forward, animationLength);
    if (!go)
        return;

    // Prepare some flags that control this algorithm.
    _abortMoveItems = false;
    this.IsMovingItems = true;

    // Move the 3D models to their new position in
    // the Viewport3D's Children collection.
    this.MoveItems_RelocateModels(itemCount, forward);

    // If we are the items host of a Selector, select the first child element.
    this.MoveItems_SelectFrontItem();

    // Start moving the models to their new locations
    // and apply the new opacity values.
    this.MoveItems_BeginAnimations(forward, animationLength);

    // Start the timer that ticks when the animations are finished.
    this.MoveItems_StartCleanupTimer(animationLength);
}

I am not going to show all of the sub-routines involved with making the magic work. If you care to see how it works, download the source code and check out the MoveItems region in the Panel3D.cs source file. However, I will show the callback method that executes when the clean-up timer ticks, because it shows how I honor the UI virtualization rule of no more than ten items in the scene at once.

/// <summary>
/// Invoked when the items stop moving, due to a call to MoveItems().
/// </summary>
void OnMoveItemsCompleted(object sender, EventArgs e)
{
    _moveItemsCompletionTimer.Stop();

    if (_abortMoveItems)
        return;

    // Remove any extra models from the scene.
    while (this.MaxVisibleModels < _viewport.ModelCount)
        _viewport.RemoveBackModel();

    this.IsMovingItems = false;

    if (0 < _moveItemsRequestQueue.Count)
    {
        MoveItemsRequest req = _moveItemsRequestQueue.Dequeue();
        this.MoveItems(req.ItemCount, req.Forward, req.AnimationLength);
    }
}

On a side note, Panel3D is smart enough to know when the user requests that it move items while it is already moving items. It maintains a queue of request structures, each representing a call made to the MoveItems method while an animation was in progress. Upon completing a request to move the items, it checks to see if any more requests arrived during the animation. If so, it initiates another call to MoveItems and starts the whole process over again.

Revision History

• May 19, 2007 — Fixed the panel so that its children can support transparency. Added the AllowTransparency and AutoAdjustOpacity properties. Updated the source code and demo application downloads. I blogged about how this change was implemented here. Thanks to Sajiv Thomas for explaining how to fix the issue!
• April 9, 2007 — After receiving some feature requests, I exposed MaxVisibleModels as a public property, added the IsMovingItems property, added the GetVisibleIndexFromChildIndex method, and added (to the demo app) the ability to bring an item to the front of the 3D scene when you click on it. I also followed Andrew Smith's advice and rewrote the logic that keeps Panel3D in sync with the selected item of the owning Selector control.
• April 8, 2007 — Created the article