I faced the same problem today and found what was the problem. You have to have your page size as Letter when you save your document to xps. Letter is 21,59cmx27,94cm and A4 is 21cmx29,97cm, so when you try to got to specific page before the document is in screen the xps is in letter format and you will "leave behind" in each page few cm. When the document is rendered visible it is back to the page size you saved the file and thus it will work normally.
I made also another solution to jump to the page without saving the document in Letter size.
DispatcherTimer dispatcherTimer = new DispatcherTimer();
dispatcherTimer.Tick += new EventHandler(dispatcherTimer_Tick);
dispatcherTimer.Interval = TimeSpan.FromMilliseconds(50);
int pagenum = 'your page';
void dispatcherTimer_Tick(object sender, EventArgs e)
if(pagenum == Document.MasterPageNumber)
Above code will jump to the page untill it is the correct page.