This is why we tell people not to store numeric items as strings: the sort order is different, and there is no way to make a string sort "properly" without converting it to a number first.
The best solution is to change your DB so it stores numbers - it may seem like work now, but it saves a heck of a lot later. But you can do it, just - it's a pain and slow, but you can.
SELECT contact_id FROM myTable ORDER BY CONVERT(INT, contact_id)
However, if any of the contact_id values is not numeric, it will fail, and return no rows.