You can't cast an array of values to any other array type because value types are "special": they aren't all the same size, and value types can't be inherited anyway.
With reference types, the variable is always the same size: 8 bytes on a 64 bit system, 4 bytes on a 32 bit. Value types can be anything between 1 byte (normally padded to a machine word boundary) and ... well, there is no absolute limit, but I'd suspect a practical limit of under 2MB as that is the default stack size for a .NET app.
Since the size of the elements isn't the same, you can't cast the array to a different type without resizing the actual elements, and that means reallocating the array and creating a new type for each element, then copying the data over. That isn't going to happen!
It may help if you see this:
Using struct and class - what's that all about?[
^] - it may help you unerstand the difference!