If you are only using a limited number of the columns, then only retrieve the ones you want: i.e. don't use
SELECT * FROM MyTable WHERE ...
List only the columns you are interested in:
SELECT Column12, Column35 FROM MyTable WHERE ...
Other than that, tuning your WHERE clause to only return the rows you want is pretty much your only other optimisation!