The effect I was looking for doesn't seem to have a commonly accepted name, but does go by dynamic grid layout
and cascading grid layout
I haven't found a nice server-side component yet, but with ryanb31's help my searches for jquery tiling
did lead me to these, and a client-side component might do the trick.
There is also a commercial component called Packery.