Don't do that: work out the angle of a line between the two points instead.
double angle(Point p1, Point p2)
{
double xDiff = p2.X - p1.X;
double yDiff = p2.Y - p1.Y;
return Math.Atan2(yDiff, xDiff) * (180 / Math.PI);
}
Then all you have to do is look at the value:
between 45 and -45 is Right, and so forth.