Click here to Skip to main content
Click here to Skip to main content
Go to top

An HTML5 Remote Desktop - Part I

, 11 May 2010
Rate this:
Please Sign up or sign in to vote.
A Remote Desktop software built on AJAX, JSON and HTML5

Introduction

VNC and Remote Desktop software were around for a long while now. Undoubtedly, they work great for most purposes but, I wanted to bring the old VNC / Remote Desktop concept over the web, to allow full web-application integration and full cross-browser cross-platform support out-of-the-box.

Despite its name, ThinVNC is not a traditional VNC, as it does not implement the AT&T RFB protocol. Instead, it is built on top of today's web standards: AJAX, JSON and HTML5.

The Code (Part I)

In this first part, we'll see the code to do the screen capture. The standard approach is to capture the whole desktop. However, in this case, we'll capture every window individually, applying clipping regions and saving the individual bitmap for later comparison and difference extracting.

Firstly, we need to enumerate all visible top windows:

TWin = class(TObject)
private
  Wnd : Hwnd;
  Rect : TRect;
  Pid : Cardinal;
public
  constructor Create(AWnd:HWND;ARect:TRect;APid:Cardinal);
end;

function EnumWindowsProc(Wnd: HWnd; const obj:TList<TWin>): Bool; export; stdcall;
var ProcessId : Cardinal;
  R,R1 : TRect;
  Win : TWin;
begin
  Result:=True;
  GetWindowThreadProcessId(Wnd,ProcessId);
  if IsWindowVisible(Wnd) and not IsIconic(wnd)then begin
    GetWindowRect(Wnd,R);
    IntersectRect(R1,R,Screen.DesktopRect);
    if not IsRectEmpty(R1) then begin
      win := TWin.Create(Wnd,R,ProcessId);
      obj.Add(win);
    end;
  end;
end;

procedure GetProcessWindowList(WinList:TList<TWin>);
begin
  WinList.Clear;
  EnumWindows(@EnumWindowsProc, Longint(WinList));
end;

We want to keep a list of windows, with their basic attributes and their bitmaps, so we can compare with the new ones and send the differences to the client. Here we merge the window list into a list of TWindowMirror:

TWindowMirror = class
private
  FIndex : Integer;
  FRgn : HRGN;
  FHandle : THandle;
  FBoundsRect : TRect;
  FProcessId : Integer;
  FImage : TBitmap;
  FDiffStreamList : TList<TImagePart>;
  ...
  ...
end;

procedure TMirrorManager.RefreshMirrorList(out OneMoved:Boolean);
  procedure GetProcessWindowList(WinList:TList<TWin>);
  begin
    WinList.Clear;
    EnumWindows(@EnumWindowsProc, Longint(WinList));
  end;

var
  wl : TList<TWin>;
  n : Integer;
  wm : TWindowMirror;
begin
  OneMoved:=False;

  wl := TList<TWin>.Create;
  try
    // Enumerates top windows
    GetProcessWindowList(wl);
    try
      for n := wl.Count - 1 downto 0 do begin
        // Looks for a cached window
        wm:=GetWindowMirror(FMirrorList,wl[n].Wnd);
       if assigned(wm) then begin
        if IsIconic(wl[n].Wnd) then
           wm.SetBoundsRect(Rect(0,0,0,0))
       else wm.SetBoundsRect(wl[n].Rect);

          // Returns true when at least one window moved
       OneMoved:=OneMoved or (DateTimeToTimeStamp(Now-wm.FMoved).time<MOVE_TIME);
        end else begin
         // Do not create a TWindowMirror for invisible windows
         if IsIconic(wl[n].Wnd) then Continue;

       wm:=TWindowMirror.Create(Self,wl[n].Wnd,wl[n].Rect, wl[n].pid);
       FMirrorList.Add(wm);
        end;
        // Saves the zIndex
        wm.FIndex:=wl.Count-n;
        // Generates clipping regions
        wm.GenRegions(wl,n);
      end;
    finally
      ClearList(wl);
    end;
    // Sorts the mirror list by zIndex
    FMirrorList.Sort;
  finally
    wl.free;
  end;
end;

Now, we do the capture:

function TWindowMirror.Capture(ANewImage:TBitmap): Boolean;
  function BitBlt(DestDC: HDC; X, Y, Width, Height: Integer; SrcDC: HDC;
                   XSrc, YSrc: Integer; Rop: DWORD): BOOL;
  begin
    // Capture only visible regions
    SelectClipRgn(DestDC,FRgn);
    result:=Windows.BitBlt(DestDC, X, Y, Width, Height, SrcDC,
                           XSrc, YSrc, Rop);
    SelectClipRgn(DestDC,0);
  end;

var
  DC : HDC;
  RasterOp,ExStyle: DWORD;
begin
  RasterOp := SRCCOPY;
  ExStyle:=GetWindowLong(FHandle, GWL_EXSTYLE);
  if (ExStyle and WS_EX_LAYERED) = WS_EX_LAYERED then
  RasterOp := SRCCOPY or CAPTUREBLT;

  DC := GetDCEx(FHandle,0,DCX_WINDOW or DCX_NORESETATTRS or DCX_CACHE);
  try
    Result:=BitBlt(ANewImage.Canvas.Handle,0,0,
    Width(FBoundsRect),Height(FBoundsRect),DC,0,0, RasterOp)
  finally
    ReleaseDC(FHandle,DC);
  end;

end;		

Now that we have captured all visible regions, we need to get the bitmap differences against the previous capture. We do this by looping through the windows, then their visible regions and finally calculating the regions where we find bitmap differences:

function TWindowMirror.CaptureDifferences(reset:boolean=false): Boolean;
  ....
begin
 ...
 result:=Capture(TmpImage);
 if result then begin
   ...
   ra:=ExtractClippingRegions(Rect(0,0,TmpImage.Width,TmpImage.Height));
   for n := 0 to Length(ra) - 1 do begin
     ra2:=GetDiffRects(FImage,TmpImage,ra[n]);
     for m := 0 to Length(ra2) - 1 do begin
       Jpg := TJpegImage.Create;
       ...
       CopyBmpToJpg(Jpg,TmpImage,ra2[m]);
       FDiffStreamList.Add(TImagePart.Create(rbmp,'jpeg'));
       Jpg.SaveToStream(FDiffStreamList[FDiffStreamList.Count-1].FStream);
       ... 
       Bitblt(FImage.Canvas.Handle,
       ra2[m].Left,ra2[m].Top,Width(ra2[m]),Height(ra2[m]),
       TmpImage.Canvas.handle, rbmp.Left,ra2[m].Top,SRCCOPY);
     end;
   end;
   ...
end;

In the next post, we'll focus on the protocol with the client and the client code. A post with updated source code can be found here.

History

ThinVNC is currently in alpha stage and this version must be taken as a technology preview. Remote mouse input is partially implemented, remote keyboard input is still missing.

License

This article, along with any associated source code and files, is licensed under The GNU General Public License (GPLv3)

Share

About the Author

Gustavo Ricardi
Chief Technology Officer Cybele Software, Inc.
United States United States
No Biography provided

Comments and Discussions

 
QuestionHelp me! Pinmemberhaigaopro29-Aug-13 6:25 
GeneralMy vote of 5 PinmemberSayehboon15-Oct-12 22:03 
very good !
GeneralI look forward to further implementations PinmemberMicroImaging13-May-10 3:58 
GeneralRe: I look forward to further implementations PinmemberGustavo Ricardi13-May-10 4:47 

General General    News News    Suggestion Suggestion    Question Question    Bug Bug    Answer Answer    Joke Joke    Rant Rant    Admin Admin   

Use Ctrl+Left/Right to switch messages, Ctrl+Up/Down to switch threads, Ctrl+Shift+Left/Right to switch pages.

| Advertise | Privacy | Mobile
Web04 | 2.8.140922.1 | Last Updated 11 May 2010
Article Copyright 2010 by Gustavo Ricardi
Everything else Copyright © CodeProject, 1999-2014
Terms of Service
Layout: fixed | fluid