How to tell what method opens a file

Question

0.00/5 (No votes)

See more:

Is there any way to tell what method has opened a file? I am using C#, and am trying to override the method used. I have the DLL injection down, and can run code from within the program itself, but the method I had been expecting was not used to open the file.

Note: The process I am hooking is not my process. I have no control of what method it uses to open the file. All I can do is override the open file command to force it into a shared state.

More Info:
A dump of dependencies can be found here: http://pastebin.com/CCkpWuq4

Though the list of initially loaded Libraries is:
wsock32.dll
kernel32.dll
user32.dll
shell32.dll
comctl32.dll
ole32.dll
oleaut.dll
version.dll
gdi32.dll
advapi32.dll
winmm.dll
msacm32.dll

Posted 7-Nov-11 12:54pm

KairuByte

Updated 8-Nov-11 5:05am

v5

Add a Solution

4 solutions

Solution 1

Read the documentation of the library you are using. Call/email the support desk of the library you are using. Use a tool such as JustDecompile[^] to decompile and read the code of each Method. Try all overridable Methods until you get the right one (the Method you want may not even be overridable). Make a random guess. At least tell us which library you are using (if it is something we might know) so we might be able to tell you.
Pick one! ;)

Posted 7-Nov-11 13:03pm

Sander Rossel

Comments

KairuByte 7-Nov-11 19:17pm

Hah,
There are unfortunately many libraries to choose from, and I have tried overriding all methods that I can think of! I won't be able to post all the libraries, but the ones I can remember are kernel32.dll, user32.dll and shell32.dll. There are no non windows default (at latest update, with .Net 4 installed) DLL's.

KairuByte 7-Nov-11 23:26pm

A dump of dependencies can be found here: http://pastebin.com/CCkpWuq4

Though the list of initially loaded Libraries is:
wsock32.dll
kernel32.dll
user32.dll
shell32.dll
comctl32.dll
ole32.dll
oleaut.dll
version.dll
gdi32.dll
advapi32.dll
winmm.dll
msacm32.dll

I'll keep looking, though I suppose there are at least a few methods that could open a file that I wouldn't think about....

Sander Rossel 8-Nov-11 2:20am

Are you going through ALL those libraries to open a file?... Why use all those calls to unmanaged code anyway? You can simply use IO.File.Open(...). See MSDN here for more information on how to open a file...
http://msdn.microsoft.com/en-us/library/system.io.file.aspx

KairuByte 8-Nov-11 8:37am

I'm not, this is an external process I am trying to hook to manipulate the file stream into a shared state instead of explicitly locking the file. I have no control over what methods have been used.

Sander Rossel 8-Nov-11 17:37pm

Well that kind of sucks for you. I do not know how to do that. As you suggested yourself it is just trial and error... Perhaps Google knows? Good luck with it though!

KairuByte 9-Nov-11 10:16am

Turns out it was CreateFile, and I just want hooking the process soon enough. Which in turn means I need to hook some sort of system process or create a driver.

Sander Rossel 9-Nov-11 13:16pm

Sounds complicated... Glad you found the Method though :)

Solution 3

Is the target a .Net Assembly? (If not, why tag the question C#?) If so, it is almost certainly using File.Open, or FileStream, if it is a read/write file. If not, it will likely be using fopen – and if it's not .Net you aren't going to be able to override anything.

You may well be best off using a decompiler, finding the file access code and changing the sharing argument in the original. That probably breaks the licence agreement but if the author is no longer interested in the game that probably doesn't matter.

Posted 8-Nov-11 4:56am

BobJanova

Comments

KairuByte 8-Nov-11 11:04am

o.0 I didn't tag it as C#, looks like someone else decided it should have been. Removing that! No, I assume the original was written in C or C++, though I can't be sure.

It is very possible to override a method in an unmanaged process, as long as it is loaded from something familiar like kernel32.dll. It requires you to inject a dll into the process and some guesswork till it's done, but it works.

BobJanova 8-Nov-11 13:12pm

Okay, in this case you need to do 'hooking'. For example, have a look at this article:
http://www.codeproject.com/KB/DLL/EasyHook64.aspx

KairuByte 8-Nov-11 13:29pm

Already am, lol. That's a part of the DLL injection I am doing.

The question has basically been answered, I need to sit and try every possible method till I find the correct one.

Bottom line, I have a lot of trial and error ahead.

Solution 2

If the application in question is opening the file with exclusive read/write, there MUST be a reason. Why are you determined to override this behavior?

Posted 8-Nov-11 2:49am

#realJSOP

Comments

KairuByte 8-Nov-11 8:53am

It is a video game that is written to only be opened once on a computer. Which makes sense for most users because normally the game is written to multiple times as new data (textures, map points, quest text, etc.) are transmitted to the game, and it's gigantic .dat file is updated. However when you make sure that all the data is downloaded beforehand (There is a command line argument), it is fairly safe to unlock the file for multiple clients.

It has been done before without issue, though the original author of that DLL has since lost interest in the game and cannot be contacted.

Add a Solution

Add your solution here

Treat my content as plain text, not as HTML

Preview 0

…

Existing Members

Sign in to your account

...or Join us

Download, Vote, Comment, Publish.

Your Email
Password
Forgot your password?

Your Email
This email is in use. Do you need your password?
Optional Password

I have read and agree to the Terms of Service and Privacy Policy
Please subscribe me to the CodeProject newsletters

When answering a question please:

Read the question carefully.
Understand that English isn't everyone's first language so be lenient of bad spelling and grammar.
If a question is poorly phrased then either ask for clarification, ignore it, or edit the question and fix the problem. Insults are not welcome.
Don't tell someone to read the manual. Chances are they have and don't get it. Provide an answer or move on to the next question.

Let's work to help developers, not make them feel stupid.

This content, along with any associated source code and files, is licensed under The Code Project Open License (CPOL)

KairuByte · Accepted Answer · 2011-11-08T07:30:00

Solution 4

Trial and error, unfortunately.

I had been hoping for an easy method identifier in the handle, or maybe an easier way.

Posted 8-Nov-11 7:30am

KairuByte