Outlook add-in integrating Skype

• Download source - 203 KB
• Download demo - 181 KB

Introduction  

We're all working with a variety of tools every day; we communicate using a number of instant messaging applications (Skype and Yahoo! for me); we deal with many emails per day using our favorite clients (mine being Outlook), and we're dealing with other tons of data for doing tracking activities, searching, and so on.

But despite this overwhelmingly increasing data, there is little interoperability between all of these programs. I know, I know, everything is in the cloud as everyone tries to sell this to us every day. But the clipboard and Alt-Tab remain the main staples of day-to-day working. I would like to set a task as completed or assign a task directly from the email received from JIRA. There may be such add-ins on market (and for Skype there are for sure), but without source code.    

The main purpose of this article is to embed Skype functionality directly in Outlook. This is not a full Skype client embedded (it demonstrates just a number of get/set properties and events), but it can be a good demonstration of the Skype Desktop API combined into an Outlook add-in. 

There are also two other purposes:   

• describes how to create a COM component in plain C (the general steps) 
• demonstrates how to implement an Outlook add-in without wizards   

For the impatient: How it looks like   

Note: Not all controls are working; some of them are for illustration purposes only. The reader will discover in code which ones are just for showing.

Background   

The familiarity with Skype Desktop API would help in order to understand what the add-in does in order to interact with Skype.

In short, there are three phases for an external application to talk to Skype:

• discover Skype and establish connection (SkypeControlAPIDiscover, SkypeControlAPIAttach)
• user confirmation in Skype to allow external application interaction
• WM_COPYDATA based data exchange with Skype following the protocol described in the Skype Developer Documentation.

Knowledge of Win32, COM, Outlook object model, and C programming are required for a better understanding of this article.

Using the code      

The sample is a single DLL written in C. While I am sure in today's world this may sound crazy, I still prefer writing in C using just the runtime the OS supplies for a number of (personal) reasons, mainly because:

1. add-in lifetime and flow are determined by their implemented contracts (in COM world, interfaces) called by their host application (Outlook),
2. the sample extensively uses COM and (especially) Win32 API, which are best dealt in the native language,
3. (almost) everything is under the developer's control, and
4. simplifies maintenance and eliminates the need for complex runtime dependencies.

The sample is a Visual Studio 2010 solution containing a single COM DLL. There are Win32 (and x64, for those with Office 2010 64-bit) configurations linked with /MT so the DLL can be simply copied and registered with regsvr32 (there is no setup program at this time). The sample has been tested with Office 2010 only (the Explorer user interface is ribbon only, no Office 2007 toolbar).  

I will also go into explaining step by step how the project is built by adding the required functionality (not only digging into code), also for the audience, which used just wizards to create COM and/or add-ins, or simply did not manually stepped through the whole process.

The project files are grouped in purpose and are usually implementing only one thing (being DLL dispatching, class factory, ribbon elements functionality, etc.). The functionality is usually packed into a structure (called in what follows object - not in the C++ sense but in the sense of a self-contained entity) which has a first member, a lpVtbl (similar to a C implementation of COM interfaces) followed by "private" member functions and data members (as in the following example from Allocator.h):   

typedef struct Allocator Allocator;

typedef struct AllocatorVtbl {
    PVOID (NDAPI *Alloc)(
        Allocator   *pAllocator, 
        ULONG       cb
        );
    ... other function members of AllocatorVtbl
} AllocatorVtbl;

struct Allocator {
    const AllocatorVtbl    *lpVtbl;

    BOOL (NDAPI *Initialize)(
        Allocator   *pAllocator, 
        HANDLE		heap
        );
    BOOL (NDAPI *Terminate)(
        Allocator   *pAllocator
        );

    HANDLE	        Heap;
#ifdef _DEBUG
    _CrtMemState    MemState;
#endif
};

The source code files are grouped into categories and subcategories based on their purpose.

• Add-in: Events (Outlook), Ribbon (hierarchically arranged), Skype I/O (general interaction with Skype), plus implementation of add-in COM objects (Addin.c Factory.c RibbonElements.c).
• Entry: DllMain and .def exports.
• Outlook: IDispatch definitions and IIDs for several Outlook objects, and a OutlookUtils.c file which deals with Outlook add-in shutdown fast behavior, version, IDispatch helpers, event advise, and type library helpers.
• Runtime: COM utils, DLL, exports, memory, resources, registry, shell helpers, string utils, threads;
• Skype: communication and utils; 
• Support: debug, security, string, and variant helpers.

Step one: COM DLL Outlook add-in   

After creating the DLL project and setting up the configurations, the usual steps are performed for creating a DLL COM component: implement DllMain, DllRegisterServer, DllUnregisterServer, DllGetClassObject, and DllCanUnloadNow (except DllMain, the rest of the four functions are from the OLNDesk.def definition file). 

DllMain (located in Entry.c file) just dispatches calls to the DllMgr object. This is a "de facto" singleton, as other objects (RegistryMgr, MemMgr, etc.) - usually those having a xxx_GetObject function returning a statically declared variable at file scope. The DLL manager manages three things:

1. the DLL_... notification calls arriving at DllMain and the HINSTANCE (the DLL part)
2. keeping track of COM references (for external objects AddRef'ed by the add-in (ExtObjRef member), the COM objects created by the DLL itself (ObjRef member) and the COM locks placed (LockRef member) by the IClassFactory LockServer call. All these are used in one of the four exported functions, DllCanUnloadNow in order to determine the number of active objects maintained by the add-in. When the number drops to 0, DllCanUnloadNow returns S_OK to the caller.     

3. DLL initialization/uninitialization itself (in this case, doing nothing).

The other four functions are implemented in the Exports.c file and similarly are just dispatchers to the RegistryMgr object (similar with DllMgr) which deals with registration (DllRegisterServer, DllUnregisterServer), object creation (DllGetClassObject), and lifetime (DllCanUnloadNow). While only the first two methods are actually dealing with the registry, I kept them all in the same structure.

The registration process does three things: checks for user to be an administrator (calling SecurityMgr IsUserAdmin) and then registering the DLL to be:

1. a COM object and
2. an Outlook add-in.

I won't insist on this process - just enumerating the (tedious) steps:

• fire up VS console and launch guidgen.exe to generate a CLSID (in source: CLSID_OLNDesk); choose also the ProgID for specifying them in add-in registration - version independent and current version (in source: ProgID_OLNDesk and VersionedProgID_OLNDesk)
• implement COM registration/deregistration in FRegistryMgr_RegisterServer_COM and Outlook add-on registration/deregistration in FRegistryMgr_RegisterServer_OutlookAddin  to be used in DllRegisterServer/DllUnregisterServer
• implement DllGetClassObject to create our IClassFactory implementation object (see below) and return it to caller
• implement DllCanUnloadNow to call DllMgr GetDllRef method discussed above and return S_OK if no active references are kept.

Additionally, RegistryMgr also implements various general registry helper functions (SetValueSz, SetValueDword, DeleteKey, CloseKey) used in other places.

The add-in implements the IClassFactory interface (in Factory.c). The implementation is standard: the only things to be mentioned are the object references maintained by the DllMgr object (increment/decrement the ObjRef on AddRef/Release calls, and LockRef in LockServer). Factory interface is created through the XClassFactory_Create function, and the add-in object (XAddin) is created inside CreateInstance.

Step two: The add-in     

The add-in object is the most important and acts in several ways:

• implements the main functionality of the Outlook add-in, mainly the _IDTExtensibility2 interface (COM add-in interface) and IRibbonExtensibility interface (for exposing Outlook user interface)
• advises Outlook objects for event interfaces and implements these interfaces
• implements Skype messages and event handlers, maintains Skype runtime information, and runs a thread for I/O message exchange with Skype.

First, the _IDTExtensibility2 interface (_IDTExtensibility.h and _IDTExtensibility.c). This interface has five methods (besides those inherited from IDispatch), and three of them represent the lifetime of our an add-in object inside Outlook. I think 99% of the dreaded Outlook shutdown problems derived from this interface are incorrect implementations (let's hope I did it right). So, OnAddInsUpdate (which notifies an add-in when Outlook add-ins have collection updates) and OnStartupComplete (which is called when the add-in is loaded during application startup) do nothing in our example.  

The other three have to consider a number of things in order to implement them correctly.

First of all, OnBeginShutdown and OnDisconnection are not called during Outlook fast shutdown. This is normally without impact on add-ins that just release COM objects inside those handlers. Our add-in has a number of other things to do, so we have to:

1. detect if Outlook will call these methods for us and
2. if not, call it manually somewhere.

1. Detection is looking for two things: if Outlook fast shutdown is enabled (OutlookMgr ReadAddinFastShutdownBehavior - remember this is enabled by default in Outlook 2010) and if the add-in specified in the installation has RequireShutdownNotification set to 0x0000001 in registry. This overrides the default fast shutdown behavior and tells Outlook "call the OnDisconnection and OnBeginShutdown methods for my add-in". I think this is a flag for supporting legacy add-ins encountering problems due to this new behavior and is likely to disappear in future versions.
2. The place where the detection test is done (and if fast shutdown is true *and* add-in does not require notifications, then call the methods) is the OnQuit method of the Outlook application event (this is advised during startup - see below).

The OnConnection method is where the add-in prepares itself. The steps done during connection are:

• Detection of Outlook version and fast shutdown behavior discussed above.
• Cleanup of temporary files left by previous add-in execution, if any (this refers to temp images generated to display Skype user picture - see below).
• Then we marshal the add-in IUnknown interface into an IStream. This is required because later we will need to access the add-in from another thread (namely, the Skype I/O thread) and this is the second common  source of Outlook add-in crashes/hangs: accessing directly an add-in COM interface from another thread than the one where it was created. Outlook objects (add-in included) are usually STA - that is, we can access them in their apartment threads. This is why we should marshal these calls into thread 0 (where all OOM sits).
• The next step is to read Skype information (application path and the executable icon, using Skype_GetAppInfo helper from SkypeUtils.c). We need Skype path and icon to launch Skype and display its icon in our ribbon.
• Then we create and start the Skype I/O thread (SkypeComm.c). This is a message loop thread which creates an invisible window of class L"OLNDesk.OutlookAddin:Windows:SkypeWatch" (kSkypeWatchWndClassName constant) for the sole purpose of exchanging Windows messages with Skype (visit the Skype Desktop API link above for the description of the message exchange protocol between an external application and Skype). The current implementation of Thread object assumes the thread object has a main window (ThreadHWND) and sends a WM_CLOSE message to this window, which in turn will end the message loop and return back in the thread function and then ends. (The thread Start creates the thread end event to be waited on termination then calls _beginthreadex to run the SThreadProc thread function; this one runs the ThreadProc which ends when the message loop ends, and finally signals the ThreadEndEvent back to our add-in. The source code is quite self-explanatory on this).   
• The next step is to store the passed Outlook's Application object into pAddin->Application (and increment the ExtObjRef member of DllMgr since we are AddRef'ing an external COM object - see DllCanUnloadNow considerations above).
• Having the Application object, now we advise for three event interfaces: application, explorers collection, inspectors collection. The usual IConnectionPointContainer/IConnectionPoint/Advise is used for all three. These events interfaces (XAppEvents, XExplorersEvents, and XInspectorsEvents) are kept in the add-in object, as well as the COM objects themselves for application (passed in OnConnection), as well as Explorers and Inspectors collections (the add-in IDispatch* members Application, Explorers, and Inspectors). All event interfaces have IDispatch implementations and rely on Invoke calls to look on dispidMember passed and invoke directly the implemented method. This is simpler than implementing an entire interface, since we can pick up in Invoke only what's interesting for us - for example, AppEvents is interesting only when Quit event arrives. (The "harvesting" of those DISPIDs corresponding to the events of interest can be done also at runtime with type library calls, however it is simpler to open MSOUTL.OLB in OleView tool, save to an .IDL file, and manually lookup for the appropriate DISPID. This article took this approach and the appropriate DISPIDs are defined inside each event interface header file, such as #define DISPID_APPEVENTS_QUIT (0x0000f007)). 

For the other two methods, their job is mainly to close/reverse what OnConnection did. OnBeginShutdown unadvises the advised event interfaces for inspectors, explorers, and application objects, while OnDisconnection releases the marshaled add-in, cleans up internal add-in structures, stops the Skype I/O thread, and waits for it to finish, and finally releases the Outlook Inspectors, Explorers, and Application objects themselves. Besides the cleanup code which is obvious, the only non-standard call here is OutlookMgr Cleanup - this frees the list of OLEnumDesc singly linked list structure. This is a structure that keeps type library information about Outlook objects and is populated in OutlookMgr FindEnumMemberByName - basically an enumerator for the MSOUTL.OLB type library, which extracts the enum constants. This is needed inside a ribbon element (see GroupSkypeStatus.c - GroupSkypeStatus_Visible) to know if the ribbon element is inside an inspector or an explorer, by looking to the OlObjectClass and comparing to the ExplorerObjectClass (value 34). 

Step three: The ribbon  

Here comes the second interface implemented by the add-in object: IRibbonExtensibility (iRibbonX member of the XAddin). The interface definition (thanks to Jensen Harris) and extensive Ken Getz documentation of ribbon elements are of a great value here. The ribbon description is simply an XML file supplied to Outlook in the GetCustomUI call - and here is where all the element types and handlers are described.

First a detail: QueryInterface implementation should supply the IRibbonExtensibility also when doing QI for IDispatch.

Then follows the implementation of IDispatch. This needs to implement GetIDsOfNames since we are supplying in ribbon a number of ribbon element handlers (OnLoad, OnGetVisible etc.) and we have to supply our DISPIDs to the caller. These will be used in the Invoke calls where we implement them. These are #define' d as constants in the IRibbonExtensibility.h file, as well as the RibbonElementType enumeration for control types (tab, group, button, and label) and RibbonControlSize enumeration for control sizes (regular and large).

So, everything being dynamic here, the GetIDsOfNames job is to map the rgszNames passed to the appropriate DISPID; Outlook will then call Invoke with the supplied DISPID. A non-exhaustive list of ribbon prototypes and arguments can be found in the _Ribbon_Prototypes_.cpp file (not used in compilation). The appropriate arguments are extracted based on DISPID and then the add-in methods (OnLoad, GetControlProperty, Action2, and OnChange) are called.

GetCustomUI completes the implementation by supplying the ribbon XML. This is where the RibbonElements.c implementation is used - mainly a collection of handlers which maintains the add-in's RibbonElements singly-linked list. All elements maintain the control ID, ribbon ID, parent ribbon ID, the function handlers (not all of them, just what is specified in the ribbon element definition), and finally the context (which is a weak pointer to the add-in object itself).

The ribbon elements are implemented in a single file (grouped under Addin/Ribbon in the project).

We resume the entire ribbon implementation logic (taking the GroupSkypeLoggedOnUser ribbon group element):  

• GetCustomUI adds the element to the list of ribbon elements

RibbonElements_Add(
    &pAddin->RibbonElements,
    RibbonID, 
    L"*",
    L"OLNDesk.OutlookAddin.GroupSkypeLoggedOnUser", 
    &GroupSkypeLoggedOnUser_Visible,
    &GroupSkypeLoggedOnUser_Label,
    NULL,
    NULL,
    NULL, 
    NULL,
    NULL,
    NULL,
    NULL, 
    NULL, 
    NULL, 
    NULL, 
    NULL, 
    (PVOID)pAddin
    );

and returns the appropriate XML section which defines the element:

//  Logged on user
L"    <group " CRLF 
L"     id=\"OLNDesk.OutlookAddin.GroupSkypeLoggedOnUser\" " CRLF
L"        getLabel=\"OnGetLabel\" " CRLF 
L"        getVisible=\"OnGetVisible\" " CRLF
L"    > " CRLF        
......
L"    </group> " CRLF

• GetIDsOfNames maps the handler name to DISPID

static
HRESULT
__stdcall
FRibbonExtensibility_GetIDsOfNames(
    IRibbonExtensibility    *This,
    REFIID                  riid,
    LPOLESTR                *rgszNames,
    UINT                    cNames,
    LCID                    lcid,
    DISPID                  *rgDispId
) {
    /*DTrace0(TRACE_LEVEL_DEBUG, 
            L"Addin/IRibbonExtensibility->GetIDsOfNames"
           );*/

    if(rgDispId != NULL) {
        UINT c;
        for(c = 0; c < cNames; c++) {
            rgDispId[c] = DISPID_UNKNOWN;

            //  ribbon methods
            ......
            else if(wcscmp(rgszNames[c], L"OnGetVisible") == 0) {
                rgDispId[c] = DISPID_RIBBONCALLBACK_ONGETVISIBLE;
            }
            else if(wcscmp(rgszNames[c], L"OnGetLabel") == 0) {
                rgDispId[c] = DISPID_RIBBONCALLBACK_ONGETLABEL;
            }
            .......
        }
    }

    return S_OK;
    UNREFERENCED_PARAMETER(riid);
    UNREFERENCED_PARAMETER(lcid);
    UNREFERENCED_PARAMETER(This);
}

• Invoke uses DISPID and the ribbon handler prototype to make the call into the add-in

static
HRESULT
__stdcall
FRibbonExtensibility_Invoke(
    IRibbonExtensibility    *This,
    DISPID                  dispIdMember,
    REFIID                  riid,
    LCID                    lcid,
    WORD                    wFlags,
    DISPPARAMS              *pDispParams,
    VARIANT                 *pVarResult,
    EXCEPINFO               *pExcepInfo,
    UINT                    *puArgErr
) {
    ......
    pAddin = IFace_GetStruct(XAddin, 
                             iRibbonX, 
                             This
                            );
    ......
    switch(dispIdMember) {

        case DISPID_RIBBONCALLBACK_ONGETVISIBLE:
        case DISPID_RIBBONCALLBACK_ONGETLABEL:
        ......
            if(pDispParams != NULL
            && pDispParams->cArgs == 1
            && V_VT(&pDispParams->rgvarg[0]) == VT_DISPATCH
            && pVarResult != NULL
            ) {
                return pAddin->GetControlProperty(pAddin, 
                                                  V_DISPATCH(&pDispParams->rgvarg[0]), 
                                                  dispIdMember, 
                                                  pVarResult
                                                 );
            }
            break;

        default:
            break;
    }
    
    return E_NOTIMPL;
}

• Finally, the add-in implements (in this case) GetControlProperty which identifies the element in RibbonElements using the ControlID and based on the DISPID (similarly to the Invoke "parent" call) invokes the RibbonElement's handler defined on creation - in these examples, Element->Visible and Element->Label - and sets the output value to be returned back to the Invoke caller.

A number of support functions (mainly for invalidating a ribbon control, or the entire ribbon) are used when a value is/need to be changed and the ribbon UI needs to be updated. These are members of another interface, IRibbonUI, defined in IRibbonUI.h. This is passed in the OnLoad ribbon handler (and cached into the add-in object) and used for invalidate/refresh the UI.

Step four: Skype I/O (and add-in corresponding elements) 

A number of Skype internal definitions are collected into SkypeDefs.h. These are documented in Skype Desktop API web page and are just enum types for various Skype constants (and are used in the implementations below).

The add-in keeps an internal structure (SkypeInfo) which holds a number of Skype properties, and a number of handlers invoked when Skype I/O needs to be done (after // Skype handlers in the Addin.h file). The communication with Skype is done by sending WM_COPYDATA messages to the Skype API window - which we discuss below. The structure members are updated either when a message is received from Skype (which changes the corresponding member structure, for example UserStatus changes when the USERSTATUS notification message is received from Skype).

SkypeWatch.c contains the implementation of Skype discovery, establishes connection, and does message exchange. This is done inside the Skype I/O thread hidden window procedure, SkypeWatchWndProc. The window procedure handles the following messages: 

• WM_CREATE. Here is where the connect timer is created and Skype messages are registered (L"SkypeControlAPIDiscover" and L"SkypeControlAPIAttach"); two twin messages (OLNdesk_msg_SkypeControlAPIAttach and OLNdesk_WM_COPYDATA) are also registered for doing PostMessage when the Skype messages are received (basically are send-to-post convertors for not blocking Skype).

The ribbon will be in the Not running state. The user can click on the Start button to launch Skype.



• WM_TIMER. Here the add-in IsSkypeRunning is called (which does a toolhelp snapshot and looks for Skype.exe). Depending on if Skype is found or not, the appropriate API status (defined in the SKYPE_API_STATUS enumeration inside the SkypeDefs.h file) is set and msg_SkypeControlAPIDiscover is broadcasted using SendMessageTimeoutW. If Skype is available, will reply with one of the SKYPE_API_STATUS values (except undefined one) using the msg_SkypeControlAPIAttach to the window we supplied in the discover message. If attach API status received is SKYPECONTROLAPI_ATTACH_SUCCESS, then in WPARAM, we get the Skype API window through which we will exchange messages from now on.  
• msg_SkypeControlAPIAttach. Received from Skype as soon as communication is established; will be posted to our window using the twin message OLNdesk_msg_SkypeControlAPIAttach.

Skype is now running and is waiting for the user to log on. The add-in status now becomes Connecting.



After the user logs on, Skype will ask for user confirmation. The add-in will remain in the Connecting status until user allows the external application to interact with Skype.



Finally, if the user clicks on Allow access in Skype, this will allow Outlook (add-in) to interact with Skype and the watch window procedure receives the new API status, which will transition from SKYPECONTROLAPI_ATTACH_PENDING_AUTHORIZATION to SKYPECONTROLAPI_ATTACH_SUCCESS.



• OLNdesk_msg_SkypeControlAPIAttach. This is where the marshaled add-in object comes in - the message unmarshals the add-in object and calls Invoke on it using the DISPID_ADDIN_SETAPISTATUS (defined in Addin.h).  This DISPID (as well as DISPID_ADDIN_PROCESSSKYPEMESSAGE) is handled in the add-in IDispatch implementation (FAddin_Invoke). The first is calling the SetSkypeApiStatus - which updates the SkypeInfo.ApiStatus (and SzApiStatus), invalidates the ribbon status elements, and performs the first read of the various Skype properties to populate the SkypeInfo members (such as balance, currency, avatar, full name, and so on).

The latter is processing Skype messages received through Skype watch window: WM_COPYDATA received from Skype is posted to the watch window using OLNdesk_WM_COPYDATA (allocating and copying the COPYDATASTRUCT received into a COPYDATASTRUCT_CTX similar structure), this message in turn retrieves again the add-in, prepares a SAFEARRAY with the COPYDATASTRUCT_CTX content, and calls Invoke on the add-in with DISPID_ADDIN_PROCESSSKYPEMESSAGE. Finally, the FXAddin_ProcessSkypeMessage retrieves the UTF-8 string sent by Skype from the unpacked SAFEARRAY, interprets the message, then parses the string, and dispatches the command.

The commands are mostly "GET <SOMETHING>" and Skype replies with "<SOMETHING> <VALUE>.

The GET AVATAR command is probably the most interesting here. We compose a temporary JPG file path and sends GET AVATAR 1 <filename>, Skype replies with AVATAR 1 <filename>, and ProcessSkypeMessage stores the path in the SkypeInfo.Avatar variable and invalidates the SkypeLoggedOnUser element. The invalidation of ribbon element will invoke the OnGetImage ribbon handler, finally being dispatched to the btnSkypeLoggedOnUser_Image call, where the image will be extracted from the JPEG file into a IPicture object using BitmapFromJPGFile. This function demonstrates the usage of CreateStreamOnHGlobal (from the JPG file content into an IStream) and OleLoadPicture to get the IPicture.

Resource loading  

The other function from RcUtils.c is BitmapFromPNGResource. This is used to get a HBITMAP from a PNG file and is used to load the Skype UI assets (a Skype developer account is needed to use them) such as presence button images (online, away etc.). This function loads the PNGs from resources of type "PNG", then uses the Windows Imaging Components (WIC) decoder interfaces (IWICBitmapDecoder, IWICBitmapFrameDecode, IWICBitmapSource) - thanks to Marius Bancila's Display images as you type in C++ sample.

Finally

The article lets various details for the user to discover. As I previously said, the purpose was to explain more how it's done, including perhaps newbie details (such as those regarding DllRegisterServer, how to make a COM manually, etc.) and less on dissecting the source code. I preferred to explain why a function is called and from where, instead of pasting large source code portions and explaining why OleCreate is called. The API details can be found on MSDN and the article could have easily transformed into bloat - anyways, more than perhaps on some places already is .

And as a personal note (unfortunately I am dealing with more than I want with such comments...). If someone wants to say/comment/ask one of these things below, save the time and read the answers:

Dear sir or madam can you do this in VB?	No.
Why did C and not <my preferred .NET language> ?	I like C better.
Can I steal your code and copy/paste to impress my overseas employer?	Maybe. Good luck with pointers. And oh, yes, all your money are belong to us.
I am doing this more easily with Shim .NET Wizards.	Is this a question?

History 

01/17/12 - Initial release.