Click here to Skip to main content
15,886,830 members
Please Sign up or sign in to vote.
4.50/5 (2 votes)
I am writing plugin for Photoshop.

Photoshop exposes C plug-in interface.

I use WPF for graphic user interface, and C++/CLI for an intermediate layer.

In one moment I call DoUI() function from C code. DoUI() is in the .cpp file compiled with /clr switch on. It means that DoUI() is compiled not to native code, but rather to MSIL. MSIL is JIT compiled in run-time when I run my plugin. However, I have a problem, and I believe it is JIT.

Plug-in works perfectly when Photoshop is not allocating large space on RAM. However, when I load large files in Photoshop and apply different memory demanding filters on them, and then try to load my plug-in FOR THE FIRST TIME - Photoshop crashes. If I load plug-in first, and then load large files and use filters - plug-in loads normally. Also, I experimented with different smaller /clr functions - instead of DoUI(). Same happens. Photoshop crashes just before calling /clr function.

So, regarding the facts that:

(1) Photoshop crashes immediately before calling first /clr function when allocated large space on RAM.
(2) Photoshop crashes only when user attempts to load plug-in FOR THE VERY FIRST TIME when large space on RAM is allocated. If plug-in is loaded before large allocations, plug-in usually loads normally after allocations. However, there is no guarantee.

I presume that there is no enough space in Photoshop memory for JIT to compile MSIL code. Photoshop crashes EXACTLY when JIT is called (calling /clr function for the first time). This explains why plug-in can be called after large allocation if it has been called before large allocation. When there are no large allocations, JIT can run successfully, store compiled code, and so the program can run without JIT after large allocations happened.

Does anybody have any advice how to control JIT behavior and what to do in this situation? How to ask Photoshop to prepare space for JIT?

ps. I tried ngen.exe. However I am not sure if Photoshop loads native cache or runs JIT.
Posted
Updated 5-Jun-15 12:26pm
v2
Comments
Sergey Alexandrovich Kryukov 5-Jun-15 18:06pm    
The problem sounds pretty difficult. (I always strongly disliked Photoshop's plug-in interface, as well as the product itself; it's good that you make your path of development in .NET.) About JIT, I would say that I don't think you can get direct access to if from your assembly. Normally, to pass through JIT, you can take into account that is normally works on per-method basis. Each method is JIT-compiled only when a method is about to be called for the very first time. It calls other methods, and they are recurrently JIT-compiled. There are cases when you need to make sure you won't face JIT-compilation on certain action. One of such cases is timing of operations. The simple trick is to call all methods in question just once before you do something else. I don't know if it can help you, so this is not an answer, just some notes.

And I'll up-vote the answer as a pretty tricky one. If your observations are correct, of course. :-)

—SA
Member 10990410 5-Jun-15 18:22pm    
Actually, I have made some progress: there is a field called bufferSpace in Photoshop plug-in interface. In the initialization phase of plug-in you can set this field so Photoshop can clean up to make room for buffers. Actually, I do not have any large buffer in my plug-in, but when I set bufferSpace to number as large as 1.000.000.000 - it actually makes some cleaning and plug-in loads correctly. (It does not really allocate 1GB it just cleans up :-) But there is no guarantee that it WILL load correctly. Photoshop must not crash. What I need is some kind of try block which would try JIT and if it fails it should catch an exception and exit. However, C++ try block does not work here, at least in my experiment.

This content, along with any associated source code and files, is licensed under The Code Project Open License (CPOL)



CodeProject, 20 Bay Street, 11th Floor Toronto, Ontario, Canada M5J 2N8 +1 (416) 849-8900