WPF Surface Panel

• Download SurfacePanelCSV2.zip - 76.52 KB

Introduction 

This article illustrates how a WPF Panel implementation can use Transforms directly for layout.
The Panel I've created for this article is called SurfacePanel and it's an attempt to create a Panel that will behave like a Windows Surface application. It's obviously not as feature rich as the real surface API but I think that it neatly illustrates how useful it can be to employ Transforms to do layout.
The example panel allows any controls to be moved around as if they're being dragged by the mouse, behaving like a physical object in the sense that they'll rotate to find the path of least resistance. The application also demonstrates how adding a drop handler can add cool effects like the contents of a folder full of images spilling out onto the panel.
It looks really cool.

Note: This is version two of this article, updated after some friendly comments from a friend and some harsh, unfriendly but (as always) accurate comments from my mentor.

Background 

In my last layout-related article I showed how to use the two-stage approach (supported by the MeasureOverride and ArrangeOverride methods) to layout child elements using the Arrange method. While this approach is sufficient, and preferred, for most layouts there exists a more powerful way of laying out controls that is more suitable when the layout requirements becomes complex.
In this article I'll use Transforms to position and rotate UIElements, I'll also use Vector3Ds to calculate new sizes for elements and because of this some portions of the article might be confusing for readers without knowledge of basic linear algebra. I'll try to keep the math section simple and short and focus on layout specifics instead but some parts will require some basic vector math.

Using the code

Download the solution, unzip and open. The solution has a class library and a WPF test app showing of a very simple sample implementation.
It's all been written using VS2010 Express Edition.

WPF Layout, standard way

In WPF layout is computed in a two-stage process where a Panel is first asked to measure and then to arrange.
During the measure pass, which kicks in when MeasureOverride is called for your Panel, the Panel is given size available to it, and it is up to the Panel to iterate over all its child UIElements and ask them how much space they need. When doing this the Panel computes an available size for each UIElement and it is up to the child to answer the question:

If I were to give you X by Y pixels, how much of that would you like to have?

The child UIElement has the right to take all of available or less than it. For example a Label will not take more than required to fit the text it has, regardless of how much space is available.
The reason for this process to be two-stage is (I think) that in some cases it makes sense to measure the children several times before arranging them. I assume this happens in the standard Grid for example if there are both *-width columns and Auto-width columns.

By calling Measure on a child element, the child element is responsible for populating it's DesiredSize property, and it's this value that's used in the ArrangeOverride method when the Rect that is the layout for the child is created. The Panel communicates the position and size by passing the Rect to the child control using the Arrange method.

WPF Layout, transform way

Using Transforms doesn't change the fundamentals of how a WPF layout is calculated, it's still a two-stage process and the Panel still have to measure and then arrange the child elements. What changes is the amount of control the Panel has over the positioning and also rotation of the child element. In the normal way the layout is specified using a Rect, that means that the Panel sets the upper left corner and the size of the child element, when using Transforms the position can be anything, and the size can be either the measured size or the measured size plus a scale transform.

What are transforms

When I've been referring to Transforms in the article so far I have been talking about the UIElement.RenderTransform property, but what is that really? The type of that property is Transform and a Transform is a high-level representation of a Matrix. The Matrix is a 2-by-2 matrix which, when multiplied by a two-dimensional vector, yields the resulting vector after the source vector has been translated (moved), skewed and scaled according to the values of the matrix. Rotation is also possible since it's techically a skewed scale. Or something like that.

The classes provided by the framework takes most of the weird math away, in most cases for example one does not need to know what the underlying Matrix is to manipulate the Transform. One important thing to note though is the fact that matrices do not commute. That means that A * B is not equal to B * A. This is important to keep in mind when manipulating Transforms since this is normally done by multiplying in translations, scales and rotations. If this is done in the wrong order the result is weird at best.

Creating the SurfacePanel

I think it's easiest to explain using an example so let's jump straight in an look at the SurfacePanel implementation.

Attached properties

In order for the SurfacePanel to be able to host any UIElement some attached properties are required to track things like position, rotation and size. This isn't something strange for a Panel implementation, the standard Grid does it with Grid.GridRow for example.
The SurfacePanel uses the attached properties SurfacePanel.PositionProperty, SurfacePanel.AngleProperty and SurfacePanel.SizeProperty to track the attributes required.

Measure pass

Since the size of each child UIElement of a SurfacePanel is stored in the SurfacePanel.SizeProperty the measure pass is simple. Because the size of the child will be set to the value of the attached property there's no point in the measure pass at all for the SurfacePanel. No point in asking the child for a desired size if you already know what size you're going to give it.

Arrange pass

The arrange pass for the SurfacePanel has two responsibilities:

• 1. Tell the child what size it is.
• 2. Position and rotate the child using a Transform.

The first point is easy, simply call the child's Arrange with a Rect containing the size. There's no point in specifying a position at this point since that will be overridden by the Transform later anyway. Also, a Rect specifies position by upper left corner, in the SurfacePanel the position of the element is the center of the element as that makes mouse driven rotations feel more natural.

The second point is also easy, create a Transform that holds the rotation and position of the child. I've used a TransformGroup which is a sub-class of Transform which allows a list of Transforms to be added. These are then multiplied together in the order added by the TransformGroup.

protected override Size ArrangeOverride(Size finalSize)
{
  foreach (UIElement child in Children)
  {
    Point position = GetPosition(child);
    Size size = GetSize(child);

    child.Arrange(new Rect(size));

    TransformGroup transform = new TransformGroup();
    transform.Children.Add(new TranslateTransform(-size.Width / 2.0, -size.Height / 2.0));
    transform.Children.Add(new RotateTransform(GetAngle(child)));
    transform.Children.Add(new TranslateTransform(position.X, position.Y));
    child.RenderTransform = transform;
  }

  return finalSize;
}

Notice how convinient and math-free this operation is, the framework provides different Transform implementations for every need so that the person implementing something does not need to know how the 2-by-2 matrix for a 40 degree rotation should look like. Sweet!

It might look weird with to translations (positioning transforms), but that's because the rotation should be around the center of the control so the first thing the transform is doing is translating it so that the center of the element is at (0, 0) by "moving it" half the width to the left and half the height up.
Second child in the TransformGroup is the rotation wich simply applies a rotation of SurfacePanel.AngleProperty degrees.
Third child is the translation that moves the child to the position specified by SurfacePanel.PositionProperty.

Grabbing a child element

Since there is no measure pass and only a simple arrange pass for the SurfacePanel it's clear that the logic for calculating size and position is located elsewhere. Because the child UIElements are manipulated using the mouse the SurfacePanel needs to figure out which child was selected on mouse down, and how to move, rotate and size that child when the move moves. Because the child controls should more or less work the way they normally do when it comes to mouse input the SurfacePanel has to use MousePreview events to inspect the input before it's handeled by the child. And because the element the mouse is over might not be a direct child of the SurfacePanel (since any UIElement may have a complex and deep visual tree) the SurfacePanel has to have a way of finding the child by traversing the visual tree:

private UIElement HitTestChildren(Point position)
{
  IInputElement inputElement = InputHitTest(position);
  if (inputElement is DependencyObject)
  {
    DependencyObject current = (DependencyObject)inputElement;
    while (current != null)
    {
      if (current is UIElement && Children.Contains((UIElement)current))
        return (UIElement)current;
      else
        current = VisualTreeHelper.GetParent(current);
    }
  }
  return null;
}

After the relevant child has been found and depending on whether the left or right mouse button was pressed (left moves and rotates, right re-sizes) some calculations are in place.

A calculated move

The elements should be have as if they were actual physical objects, grabbed at the mouse position when moved. This means that they first rotate so that the contact point is maintained and then moved to cover the distance that rotating cannot account for.

If the mouse was moved from the green point to the red point the angle the UIElement has to rotate is equal to the angle between the two vectors A and B (where A is the vector from the center of the element to the start point and B is the vector from the center of the element to the current point).
In the past when I've calculated the angle between two vectors I have taken arccos for the dot-product of A and B divided by the absolute values of A and B multiplied with each other.
But then WPF came along with a static method called Vector.AngleBetween(Vector, Vector) and that seemed alot nicer to use :).

Vector centerToPrevious = previousMousePosition - selectedPosition;
Vector centerToCurrent = currentMousePosition - selectedPosition;

double angle = Vector.AngleBetween(centerToPrevious, centerToCurrent);
TransformGroup transform = (TransformGroup)selectedChild.RenderTransform;
double newAngle = ((RotateTransform)transform.Children[1]).Angle + angle;

The move is slightly more complicated to calculate. If Shift is down while dragging then that means no rotation, move only and the new position is simply old position plus the dragged distance (i.e. the vector between the start and current point). If Shift is not down current position is moved to the current position plus the difference in distance between A and B along B.

if (KeyboardHelper.IsAnyShiftDown)
  selectedPosition += currentMousePosition - previousMousePosition;
else
  selectedPosition += VectorExtensions.Normalize(centerToCurrent) * (centerToCurrent.Length - centerToPrevious.Length);

Size does matter

Size, it turns out, is the hardest thing to calculate. It would be simple if it wasn't for the rotation; if I drag the mouse up I want the element's height to increase, but only if it's not rotated. If it is rotated 45 degrees and I drag straight up I want the height to increase by half of what it would if there was no rotation, but I also want the width to increase. This means that each element, when sizing, must be aware not only of mouse drag direction, but also what is up and down and left and right not to the SurfacePanel but to itself. Regardless of current rotation. This is where some weird vector math kicks in.
The difference in height is equal to the difference in B projected onto V, then subtract from that A projected onto V where V is the (0, 1) vector rotated by the rotation of the element. And the same goes for width but with H in place of V.
To calculate the vector projection I've implemented an extension method:

public static class VectorExtensions
{
  public static Vector Normalize(Vector v)
  {
    Vector result = v;
    result.Normalize();
    return result;
  }

  public static Vector Project(this Vector a, Vector b)
  {
    double angle = Vector.AngleBetween(a, b).ToRadians();
    return Normalize(b) * a.Length * Math.Cos(angle);
  }
}

The actual size calculation method, along with stuff to hangle elements that need their aspect ratio maintained, then becomes:

private void CalculateSize(Vector centerToPrevious, Vector centerToCurrent, double angle)
{
  Vector vertical = new Vector(Math.Sin(angle), Math.Cos(angle));
  Vector horizontal = new Vector(-vertical.Y, vertical.X);

  double deltaY = centerToCurrent.Project(vertical).Length - 
                  centerToPrevious.Project(vertical).Length;
  double deltaX = centerToCurrent.Project(horizontal).Length - 
                  centerToPrevious.Project(horizontal).Length;

  Size size = GetSize(selectedChild);

  // Multiply by two to get uniform resize
  size.Height = Math.Max(MinSize, size.Height + deltaY * 2.0);
  size.Width = Math.Max(MinSize, size.Width + deltaX * 2.0);

  if (GetMaintainAspectRatio(selectedChild) || KeyboardHelper.IsAnyCtrlDown)
  {
    double ratio = size.Height / size.Width;
    if (Math.Abs(deltaY) > Math.Abs(deltaX))
      size.Width = size.Height / ratio;
    else
      size.Height = size.Width * ratio;
  }
  SetSize(selectedChild, size);

  InvalidateMeasure();
  InvalidateVisual();
}

Move, Rotate and Size

Full implementation of the mouse down and mouse move method:

private void SurfacePanel_PreviewMouseDown(object sender, MouseButtonEventArgs e)
{
  Point position = e.GetPosition(this);
  UIElement child = HitTestChildren(position);
  if (child != null)
  {
    Children.Remove(child);
    Children.Add(child);
    selectedChild = child;
    selectedPosition = (Vector)GetPosition(selectedChild);
    SetVelocity(selectedChild, new Vector());
    SetAngularVelocity(selectedChild, 0);
    previousMousePosition = (Vector)position;
  }
}

private void SurfacePanel_PreviewMouseMove(object sender, MouseEventArgs e)
{
  currentMousePosition = (Vector)e.GetPosition(this);
  if (selectedChild != null && 
      (e.LeftButton == MouseButtonState.Pressed || e.RightButton == MouseButtonState.Pressed))
  {
    Vector centerToPrevious = previousMousePosition - selectedPosition;
    Vector centerToCurrent = currentMousePosition - selectedPosition;

    double angle = Vector.AngleBetween(centerToPrevious, centerToCurrent);
    TransformGroup transform = (TransformGroup)selectedChild.RenderTransform;
    double newAngle = ((RotateTransform)transform.Children[1]).Angle + angle;

    if (e.LeftButton == MouseButtonState.Pressed)
    {
      if (!KeyboardHelper.IsAnyShiftDown)
      {
        SetRotationTransform(selectedChild, newAngle, transform);
        SetAngularVelocity(selectedChild, angle);
      }

      if (!KeyboardHelper.IsAnyCtrlDown)
      {
        if (KeyboardHelper.IsAnyShiftDown)
          selectedPosition += currentMousePosition - previousMousePosition;
        else
          selectedPosition += VectorExtensions.Normalize(centerToCurrent) * 
                              (centerToCurrent.Length - centerToPrevious.Length);

        SetPositionTransform(selectedChild, selectedPosition, transform);
        SetVelocity(selectedChild, currentMousePosition - previousMousePosition);
      }
    }

    if (e.RightButton == MouseButtonState.Pressed)
      CalculateSize(centerToPrevious, centerToCurrent, newAngle.ToRadians());

  }
  previousMousePosition = currentMousePosition;
}

The SetAngularVelocity and SetVelocity are methods that store the current delta as attached properties which allows for friction based movement to take place after the user release the element.
I've covered animation of WPF panels before so I won't go into that in this article.

Peripheral stuff

Drop handlers

In order to be able to drag an image or indeed an entire folder of images onto the SurfacePanel some drop handlers are required. In this solution a drop handler is simply a class that enables and hooks into the Drop event of the SurfacePanel.
The handler for dropping images, ImageDropHandler, for example simply inspects the drop information and creates Image elements from every .jpg file it finds.

  public void Attach()
  {
    panel.AllowDrop = true;
    panel.Drop += HandleDrop;
  }

  public void Detatch()
  {
    panel.AllowDrop = originalAllowDrop;
    panel.Drop -= HandleDrop;
  }

  private void HandleDrop(object sender, DragEventArgs e)
  {
    IEnumerable<FileInfo> files = ExtractFileList(e);

    Queue<Stream>
    streams = new Queue<Stream>();
    AutoResetEvent resetEvent = new AutoResetEvent(false);

    BackgroundWorker loader = new BackgroundWorker();
    BackgroundWorker adder = new BackgroundWorker();

    loader.DoWork += LoadImages;
    adder.DoWork += AddImages;

    ParameterType parameters = Tuple.Create(files, streams, resetEvent);
    loadingDone = false;
    loader.RunWorkerAsync(parameters);
    adder.RunWorkerAsync(parameters);
  }

  private static IList<FileInfo> ExtractFileList(DragEventArgs e)
  {
    IList<FileInfo> files = new List<FileInfo>();

    if (e.Data.GetFormats().Contains(DataFormats.FileDrop) && e.Data.GetDataPresent(DataFormats.FileDrop))
    {
      foreach (string fileName in (string[])e.Data.GetData(DataFormats.FileDrop))
      {
        DirectoryInfo directory = new DirectoryInfo(fileName);
        if (directory.Exists)
        {
          foreach (FileInfo file in directory.GetFiles("*.jpg"))
            files.Add(file);
        }
        else
        {
          FileInfo file = new FileInfo(fileName);
          if (file.Name.ToLower().EndsWith(".jpg"))
            files.Add(file);
        }
      }
    }
    return files;
  }

The loader and adder BackgroundWorkers is only a way to load multiple images while allowing the UI to remain responsive using as little time on the Dispatcher's thread as possible.

Sorters

The SurfacePanel supports sorting of it's elements by setting the dependency property Sorter to a ISurfaceSorter instance. That means that whenever a Sorter is not null the animation engine of SurfacePanel will animate to the sorted position instead of the friction based one.

The implementation of the StackSorter used for the text example in the video looks like this:

  public class StackSorter : ISurfaceSorter
  {
    private readonly IComparer<UIElement> comparer;

    public StackSorter()
    {
    }

    public StackSorter(IComparer<UIElement> comparer)
    {
      this.comparer = comparer;
    }

    public bool CalculateSorted(SurfacePanel panel, UIElement selectedChild, double elapsedTime)
    {
      if (panel.Children.Count == 0)
        return true;

      IEnumerable<Size> sizes = from e in panel.Children.Cast<UIElement>() select SurfacePanel.GetSize(e);

      double maxHeight = (from s in sizes select s.Height).Max();

      double x = 0;
      double y = 0;

      foreach (UIElement element in panel.Children.Cast<UIElement>().Where(i => i != selectedChild).OrderBy(i => i, comparer ?? new ChildIndexComparer(panel)))
      {
        Size size = SurfacePanel.GetSize(element);
        Vector position = (Vector)SurfacePanel.GetPosition(element);

        if (x + size.Width > panel.ActualWidth)
        {
          x = 0;
          y += maxHeight;
        }

        Vector targetPosition = new Vector(x + size.Width / 2.0, y + size.Height / 2);
        x += size.Width;

        Vector direction = (targetPosition - position) * 0.2 * elapsedTime;

        double angle = SurfacePanel.GetAngle(element);
        SurfacePanel.SetAngle(element, angle - angle * elapsedTime);
        SurfacePanel.SetPosition(element, (Point)(position + direction));
      }

      return false;
    }
  }

Points of Interest

The YouTube video of the sample application looks really cool I think, have a look if you haven't already done so.

History 

• 2011-03-13; First version
• 2011-03-25; Second version, fixed ZIndex, Aspect Ratio, use of FrameworkElement Min size and (most importantly) rotations on scaled objects.