It is impossible to delete any file loaded in memory for execution. The only way to delete it is to kill the process which uses the file, say, with
TerminateProcess:
https://msdn.microsoft.com/en-us/library/windows/desktop/ms686714%28v=vs.85%29.aspx[
^].
But beware: if you have a need to delete such file, it's a pretty certain indication of
some kind of misuse. If you really have a need to delete a file which is an executable module used by some application and cannot do it, it means that the application has some bugs which you have to fix. Or the application is not designed to be terminated at that moment, so you should not delete any file it uses. And so on…
[EDIT]
By the way, your understanding of how EXE works is wrong: "exe file is copied into memory and is executed". If you did it, you could not start the process based on the application code. Application is
loaded by an important part of software, the
loader, which does a number of sophisticated things. One important part of it is: on systems based on x86 architecture in
protected mode, you cannot execute any piece of code loaded in memory in the application mode at all, due to hardware-based
execution prevention; preparation of the piece of code memory is one of the
kernel-mode operations not allowed in the outer
ring.
—SA
Submit an article or tip