Video streaming usually uses the
Real-Time Streaming Protocol[
^]. As Richard pointed out, UDP will force you to handle packets that get lost or arrive out of order. And TCP is too inefficient for streaming. RTSP was invented to address these problems.
Another option is the
Real-Time Transport Protocol[
^] (RTP).
You might be able to find an open-source implementation of either of these if you look around on GitHub. Failing that, you'd have to implement one yourself, but you were planning to do that kind of work anyway. This way, at least you'll have a standard to follow.