If you use a debugger, you can always stop at the point before the call to the function, not just inside this function. This way, you can see where the control goes instead of the call you need.
Also, your bug is probably
lexical. JavaScript does not interpret code line by line from the very beginning; if first-run parsing and lexical analysis fails, even correct lines won't be executed; and the usual exception handling won't help you to detect it. This statement may sound surprising to you, but it's hard to check-up :-).
Look at the line #5, first loop in your code. The { bracket is not closed, it looks like. One method of detecting lexical bugs is described in this chapter of my article:
JavaScript Calculator 5. Handling Lexical Errors.
By the way, the algorithm, as well as many others, is implemented in JavaScript here:
https://code.google.com/p/crypto-js.
[EDIT]
See also my comment to the question. For cryptographic purposes, MD5 or SHA-1 are unreliable, as these algorithms are broken. For such purposes, one should rather use one from the SHA-2 family, or SHA-3:
https://en.wikipedia.org/wiki/Cryptographic_hash_function[
^],
https://en.wikipedia.org/wiki/MD5[
^],
https://en.wikipedia.org/wiki/SHA-1[
^],
https://en.wikipedia.org/wiki/SHA-2[
^],
https://en.wikipedia.org/wiki/SHA-3[
^].
—SA