Click here to Skip to main content
14,365,158 members
Rate this:
Please Sign up or sign in to vote.
The situation (all parts of program are built using C#):

The program MUST run on a CLIENT PC. A user will manually login to the Host using windows built in Remote Desktop Connection. Only one Remote Connection will be open at any given time.

I am curious if there is a way for the program running on the Client PC to "see" windows/msg-boxes that appear in the Host-Remote Desktop Window. The Client PC program does not need to actually interact with the Host-RDC, it only needs to see when the events happen. When the Client-Program sees the specified events, it will send communication to the user so that new tasks can be started.

I have read multiple threads on here talking about how to get a system to log in, such as
Remote Desktop using C#.NET
Palantir - Remote Desktop Manager

but, I have been very unlucky in finding a way to interact or see events that happen within the Host-RDC window.

Essentially, the program must be running on the Client system and will look for certain windows to appear within in the Host-RDC window. If the Client-PC program notices a set of specific windows within the Host-RDC window, the Client-PC program will initiate alerts/flags.

Thank you for any help that can be provided! It is greatly appreciated, even if it is a directly link pointing me to an article I missed!
Posted
Updated 16-Apr-14 4:48am
v6
Comments
Sergey Alexandrovich Kryukov 15-Apr-14 14:32pm
   
Are you sure you really need RDP, not some custom protocol to fit your needs?
Besides, you application can collaborate on the network the way you want, so you can work with it from a remote host.
—SA
AsmoconX 15-Apr-14 14:36pm
   
There are tests that are being ran on the system. We have a dedicated PC, that uses a direct-lan cable plug into the Test-PC (the Test-PC does not have monitor options or other input options). The RD is logged into, so that tests may be ran on that Test-PC. The objective, is to try and relay information out to the dedicated-Host PC to alert people when tests complete or other various tasks.
Sergey Alexandrovich Kryukov 15-Apr-14 14:38pm
   
Whats wrong with available RDP?
—SA
AsmoconX 15-Apr-14 14:44pm
   
I do not think anything is wrong with RDP, but I do know understand how to allow the program to see or interact with the elements inside the remote desktop window. Every time I try, or use Spy++, it will only give me information about the main RDC window, but no information about the children within it. That's what I am hoping to find out, is if there is a way to see (monitor) the objects within the RDC window.
Sergey Alexandrovich Kryukov 15-Apr-14 16:30pm
   
I'm missing something. Here is what I mean: imagine you simple use available RDP services and available RDP clients, from Microsoft. What are the feature which are missing in this scenario? What are you going to do to control applications through RDP?

And, if such missing features is the case, the answers is: you have to develop specialized application, designed for some kind of collaboration. The whole idea if RDP does not fit here. You are right: there is no access to individual application windows through RDP. RPD is pretty simple and easy: it simply makes raw events passing through a middleman transparently; it does not "know" anything about applications and their windows...

—SA
AsmoconX 15-Apr-14 17:14pm
   
Test PC -- Uses Remote Desktop Connection to Connect to Remote PC. The Remote PC has specialized files already on it, that must be ran to validate configurations and components.

The person on the Test-PC is controlling all of the tests to run via the Remote Desktop Connection (using mouse clicks and keyboard commands, etc). The hope, is that the person could start a program on the Test-PC that will monitor the actions happening inside that Remote Desktop Window, so that if a test is finished [during test completion a Event Window is displayed], the program will detect that Event Window inside the Remote desktop Window and cause an event notification back on the TestPC.

{TestPC}-----------(remote login)----------{Remote PC}
Program Runs --------------------------- Event Happens
Programs Detects Event ----------------- (no changes)
Program Sends Event Notification ----- (no changes)
User Interacts ---------------------------- Event is manually cleared by User
Sergey Alexandrovich Kryukov 15-Apr-14 17:18pm
   
I already explained, this is not how RDP works. It just transparently passes raw events. RDP "knows" nothing about the windows inside the desktop (client), or windows inside RDP window (client). The whole idea is not helpful.
—SA
AsmoconX 15-Apr-14 17:29pm
   
So, your comment of there is no way to interact or relay this information without having a specialized program is essentially the answer. :( Since the Remote system can not be modified, I do not have a way to communicate the window information back to the Test-PC.

This part was unknown and all other code was finished (logic for alerts, sending notifications, etc, etc). Feels crazy that there is no way to interpret events within the RDC window. :(
Sergey Alexandrovich Kryukov 15-Apr-14 17:35pm
   
It depends on your goal. You cannot do it the way you suggested. If you explain the ultimate goals, we can discuss the problem. Not having access to the server-side computers (those you want to control) is a huge limitation. It's almost like saying: "I have some computers not connected to the network; how can I control them remotely"? They have RDP services installed (by they way, if you don't have control, even they can be disabled, you may have no access at all. Perhaps "nothing else can be added to or modified on the remote PC side" means that you can enabled RDP and gave a member access to those computers. If that, why nothing else.

I suspect, your main problem is that your goals are not properly set. You should explain why do you need some control over those computers, what can if give you.

—SA
AsmoconX 15-Apr-14 17:59pm
   
This is a test environment and information is not meant to be added or modified on the remote systems (for validation and testing requirements). The test-process requires us to login to the system using a Remote Desktop Connection and perform a set of tests within that Remote Desktop Environment (Again, the remote system does not have other connection methods such as USB, PS2, VGA, etc..Just a console network line plug). The remote PCs already have the operating system installed, configured, and have specialized test-programs pre-loaded onto them that must be ran for validation purposes.

After we login to the remote system, we perform multiple days worth of tests using the software already on the remote system.

The end-goal was simply to find a way to let individuals know when a test has completed or had an error. The program would be able to detect that completion/error window within the remote desktop window and turn on a light, send an email to the operator, and various other things.

There can be 20 different systems testing like this and each Test-PC must be monitored to see when it is time for a new action. So, the final objective is to reduce the time lost because a person is unable to go turn on each monitor, check the remote display, act if needed or turn display back off and continue check the other displays. If the event could be relayed, like I was saying through a program that was monitoring the remote window, they would not be occupied checking systems that may not even require attention yet.

I honestly do not know how else to explain it. :(
Sergey Alexandrovich Kryukov 15-Apr-14 18:02pm
   
Oh, now this is sufficient explanation which is clear enough. Very good, thank you. All right, I'll put my answer. Please wait.
—SA
Sergey Alexandrovich Kryukov 15-Apr-14 18:38pm
   
I answered in detail, please see Solution 1.
By the way, after your last clarifications, I reconsidered your question as quite interesting and up-voted it.
—SA
Rate this:
Please Sign up or sign in to vote.

Solution 1

Why not create a server / client program set? It's not too difficult to create a server client relationship using TCP or some other protocol other than RDP. The 'client' can monitor pretty much any activity on the host machine (as well as control it) and report back to the 'server'.
   
Comments
AsmoconX 15-Apr-14 14:58pm
   
James_Finch: The main thing that causes a problem is that nothing else can be added to or modified on the remote PC side.

Basically: Person connects a direct LAN cable from Test-PC to Remote-PC. Person uses RDC to login to the Remote-PC and specialized tests are ran to validate integrity (tests are pre-built into the Remote PC).

No software or modifications can happen on that Remote-PC, so my hopes was to have this program run on the Test-PC and monitor windows/alerts/failures/etc on the Remote-PC so that issues can addressed quicker in a busy environment.

The objective of this, is simply to allow the person running the tests through the RDC to have an alert when something finishes or a problem arises.
Sergey Alexandrovich Kryukov 15-Apr-14 16:32pm
   
Which side do you mean by "remote PC side"? Is it a side of RDP service (the computers to be controlled)? Please see my last comment. If you hope to get into application detail via RDP, it won't work.
So, if you cannot change anything on those computers, you don't have enough controls over them.
—SA
Sergey Alexandrovich Kryukov 15-Apr-14 20:12pm
   
OP finally clarified the question: the problem is very different. Please see Solution 2.
—SA
Rate this:
Please Sign up or sign in to vote.

Solution 2

AsmoconX wrote:
This is a test environment and information is not meant to be added or modified on the remote systems (for validation and testing requirements). The test-process requires us to login to the system using a Remote Desktop Connection and perform a set of tests within that Remote Desktop Environment (Again, the remote system does not have other connection methods such as USB, PS2, VGA, etc..Just a console network line plug). The remote PCs already have the operating system installed, configured, and have specialized test-programs pre-loaded onto them that must be ran for validation purposes.

After we login to the remote system, we perform multiple days worth of tests using the software already on the remote system.

The end-goal was simply to find a way to let individuals know when a test has completed or had an error. The program would be able to detect that completion/error window within the remote desktop window and turn on a light, send an email to the operator, and various other things.

There can be 20 different systems testing like this and each Test-PC must be monitored to see when it is time for a new action. So, the final objective is to reduce the time lost because a person is unable to go turn on each monitor, check the remote display, act if needed or turn display back off and continue check the other displays. If the event could be relayed, like I was saying through a program that was monitoring the remote window, they would not be occupied checking systems that may not even require attention yet.

I honestly do not know how else to explain it. :(
Now, the problem is properly explained, thank you.

Well, the whole testing scenario simple questionable to me, but I can imagine setting where such scenarios have a write to exist. I hoped to find some available products for this kind of testing, but it's going to take a lot of time. Please see: http://en.wikipedia.org/wiki/Test_automation#Graphical_User_Interface_.28GUI.29_testing[^].

Review this list of tools: http://en.wikipedia.org/wiki/List_of_GUI_testing_tools[^].

Perhaps you may need to test some of them and evaluate.

It's good to get more familiar with many different approaches to testing. See also:
http://en.wikipedia.org/wiki/Software_testing[^].

Now, as I cannot give you my final verdict on using any of the particular available tools, I'll explain how the tool you need has to work. It will also explain to you how can you develop such tool on your own. It is not going to be easy, but I've done such things, they are doable. I'll start with the simplest scenario.

Here is the idea: as a user, you can connect using RDP (in your case, you should not and need not create anything custom) and do all the testing manually. Then you need to repeat the test many times. The only problem that the human operator cannot repeat it many times, especially if you need to do it consistently (I say, this is still the simplest scenario). Now, imagine that you have some spy application which records all the system-global mouse and keyboard events and can play them back on your whole testing system, to complete all the test cycle from the beginning to the end. Some of those events will go to RDP client window, and RDP will transparently pass them to the computer under the test, and that remote computer will reproduce all the actions of the human operator.

How to implement it? Not very easy, but quite doable. First of all, recording. You will need to use Windows Hooks. Please start here: http://msdn.microsoft.com/en-us/library/windows/desktop/ms632589%28v=vs.85%29.aspx[^].

There are two approaches:
  • You can use just the local hooks. This is much simpler than the global hook. You need to set the dwThreadId; and this should be the thread of the RDP client. The biggest simplification is: you can do it all in .NET, using P/Invoke:
    http://en.wikipedia.org/wiki/P/Invoke[^],
    http://msdn.microsoft.com/library/en-us/vcmxspec/html/vcmg_PlatformInvocationServices.asp[^].

    One considerable benefit of local hooks is: all the events unrelated to your RDP client go out of equation.

    A little problem is: it may require a custom RDP client which can host the hook in its code, which is actually good for development and testing. This is not a biggest problem: you are already familiar with RDP clients, don't you? :-)


  • Nothing like that can be done with global hooks. If you specify no thread, the hook will be system-global. They are hard to develop, event harder to test. And you cannot do it all in .NET. You will have to develop some native (unmanaged) DLL setting the hook and doing hook handling. According to Microsoft documentation, such DLL is required if the hook is global. You will need to load this DLL in your .NET code and communicate with the hook functions using some thread synchronization mechanism.


In all cases, the hooks should detect all the mouse and keyboard events (only related to a RDP client, in first approach shown above) and record those events in some form in some file. Testing, essentially, will be playing them back. (Still, we are at the simplest scenario.)

There is only one way of playing those events back: the raw Windows API SendInput:
http://msdn.microsoft.com/en-us/library/windows/desktop/ms646310%28v=vs.85%29.aspx[^].

You can use it in your .NET code using P/Invoke, but everything is already done for you: http://www.pinvoke.net/default.aspx/user32.sendinput[^].

Now, how about the complex scenario? You may need to create more complex tests, when the user events are not just recorded, but also modified during testing based on some calculation. Also, you may need to modify timing, to improve performance of the test. If can create one very serious problem: you might feed the simulated events to the system faster than they can be handled on a remote computer, with all the time lags, especially network traffic. When it happens, the test is totally broken: you may send events to non-existing controls, or controls not located where you click with the mouse, and so on. Well, the test itself needs to be tested somehow, but the accelerated test may become unreliable, because of some possible traffic gems on the network, and the like.

In essence, you may need to have not a user-mocking machine, but some intellectual testing robot. Well, then you would need to develop some language of testing scenarios based on key/mouse event primitives, combine recorded event sequences using some branching, conditional or not, looping or not, and a lot more. Good work to be done.

So, first of all, review some available UI testing tools.

—SA
   
v3
Comments
AsmoconX 16-Apr-14 9:40am
   
Sergey,

Thank you for all the time put into your reply. I have been reading a lot of the links and looking into the Windows Hooks methods.

On a side note, I realized a lot of the confusion from my original post was due to wording. I changed the original post to use the correct Host/Client terms, so that if anyone else reads it, it will make more sense for what I was doing. Thank you for pointing that out!

I do not know if I will actually get this resolved anytime soon, but again, I appreciate your time.
Sergey Alexandrovich Kryukov 16-Apr-14 11:42am
   
You are very welcome.

Very good. Again, you may want to check out available UI testing tools first.
Are you going to accept this answer formally (green "Accept" button)? In all cases, your follow-up questions will be welcome.

—SA
AsmoconX 21-Apr-14 12:07pm
   
I have accepted your solution. I looked at the available UI testing tools, but that is going against what needs to be accomplished.

Just to give more information: The HOST-PC is a controller for a very large array of hardware. The Client-PC can only login to the HOST-PC to run specific tests, from the HOST. Again, no other software or hardware can be used on the HOST due to security reasons, etc, etc, etc.

That is why I was hoping to find a way to have the client-PC be able to interpret the windows inside the remote desktop window, so that people could be alerted to when one of the tests completes or needs action. The testing software itself, can not be modified or adjusted either, since it is specifically built to run very detailed hardware, i/o, path, redundancy, etc, tests.

Thank you again...Hopefully I will be able to make this work one of these days!
Sergey Alexandrovich Kryukov 21-Apr-14 14:40pm
   
Than you for this note. Well, creation of your own testing tools remains an opportunity.
Good luck, call again.
—SA

This content, along with any associated source code and files, is licensed under The Code Project Open License (CPOL)




CodeProject, 503-250 Ferrand Drive Toronto Ontario, M3C 3G8 Canada +1 416-849-8900 x 100