I would say,
byte[] newbuffer = new byte[newLength];
Array.Copy(buffer, newFirstIndex, newbuffer, 0, newLength);
buffer = newbuffer;
I think that's all you can do; you can't arbitrarily "deallocate" trailing and leading parts, even using unsafe methods.
—SA