Why have you neglected to do the research yourself on these Virtual Box API calls? I have never used the Virtual Box API but it took less than 10 seconds to find the SDK reference using my favorite search engine.
Virtual Box IDisplay
I do realize that codeproject.com is the coolest orange website on the planet but you would probably get great advice from software engineers that have experience using this API from the virtual box forum.
Virtual Box Forum
I am guessing that you should do the following:
1.) Call getScreenResolution and get the width, height and bit depth.
2.) Change your BYTE
into a LPBYTE
and allocate the space required for the bitmap using width,height,bit depth.
3.) Call takeScreenShot or takeScreenShotToArray or perhaps takeScreenShotPNGToArray to get the bitmap bits.
Some caveats: It looks like takeScreenShot and takeScreenShotToArray always returns 32 bit bitmap bits based on SDK description. It also looks like takeScreenShotPNGToArray might be the best choice, you might even be able to dump the returned buffer into a file.