Using UPnP for Programmatic Port Forwardings and NAT Traversal

• Download the PortForward demo project (includes a release-build executable that you can run without the need to compile) - 131 Kb
• Download an installation package (installs the PortForward utility, together with a user-oriented "Help" file) - 462 Kb
• Source code for the PortForward engine only - 11.5 Kb

Contents

• Introduction
• Requirements for UPnP
• The debate: Convenience vs. Security
• The demo program
  • The engine
  • The UI
• Installation package
• A quick rant
• Bibliography and reference materials
• License information
• History

Introduction

Universal Plug-n-Play ("UPnP") is an attempt to extend the concept of ordinary plug-n-play, so that it applies to more than just your own machine: it applies to the whole network. For example, with ordinary plug-n-play, when a new peripheral is connected to your machine, it is automatically discovered and configured from your machine without access to the peripheral itself. UPnP extends this idea to the network: when a new network device is connected to the network, it can be automatically discovered over the network, and configured remotely from your machine over the network.

The idea is that a device can dynamically join a wired or wireless network, obtain an IP address, convey its capabilities, and learn about the presence and capabilities of other devices all over the network. UPnP envisions a future where all devices are networkable and controllable over the network, such as light switches, thermostats, toasters, automobiles, etc. More information can be found here.

Since 2002, most routers have UPnP capability. This allows you to solve one of the more vexing problems for users of network programs that must accept an incoming connection from the Internet. Examples of these programs include P2P file sharing programs, interactive games and gaming, video conferencing, and web or proxy servers. To allow others on the Internet to connect to these programs, it is necessary to configure the router to accept incoming connections and to route the connection to a local machine on the LAN behind the router. This process is called "port forwarding" or "NAT traversal" ("Network Address Translation").

For the ordinary user, this process can be daunting. Moreover, it's often not easy to explain how to configure a router for port forwarding, for the reason that the method differs for each different type of router by each different manufacturer.

UPnP works perfectly in this situation. With it, you can map a port-forwarding programmatically without user interaction.

This article describes a utility that discovers current port mappings on a UPnP-enabled router, and allows you to add/edit/delete mappings. The utility is conceptually broken into two pieces: an engine that performs the actual work, and the UI that uses the engine. This way, it should be possible for you to re-use the engine for your own purposes.

go back to top

Requirements for UPnP

To be able to remotely configure a router over a network from a local machine, you need the following:

• UPnP on the local machine: Basically, you must have Windows® XP, any service pack. Older versions of Windows, including the very-popular Windows® 2000, will not work, as they do not have UPnP capability. It might also be necessary to enable UPnP, since a UPnP-capable OS does not necessarily have it turned on by default.
• UPnP on your router: Most routers manufactured since 2002 will have UPnP capability. Again, it might be necessary to enable UPnP on the router, since it might not be turned on by default.

In addition, if there is a firewall on the local machine, it must be configured to allow the underlying TCP and UDP communications on which UPnP relies. Specifically, it must be configured to pass TCP port 2869 and UDP port 1900.

go back to top

The debate: Convenience vs. Security

Before going on, it's worthwhile to point out the current debate over whether it makes any sense to include UPnP capability on a router.

Proponents argue based on convenience: It's hard to configure a router for port forwarding, and average users are not able to do it easily. There are different methods for each different router, and even if you can find the correct method, understanding it requires the user to learn confusing network terminology.

Opponents argue that there's a significant security risk: Routers insulate local networks from the wilds of the Internet, by blocking incoming connections that are almost always malicious. This insulation is at the hardware level, and as such, it is often more effective than software such as software firewalls. Most users rely on this added layer of insulation to protect machines on their local network, and the average user relies on it without even being aware of it. But since UPnP allows any program, even malicious programs, to create a port mapping through the router, this added layer of insulation disappears. Moreover, with UPnP, the port mapping can be created even without any knowledge of the administrative password to the router, and thus can be created without the knowledge or consent of the user.

For me, I side with the opponents of UPnP for routers. The added layer of security is a true benefit, and it benefits those most likely to need it (i.e., relatively unsophisticated users who are most often targeted by malicious programs). Moreover, although manual configuration of port mappings is complicated, the programs that absolutely need it are also complicated, and require a relatively higher level of user sophistication anyway. Finally, there are many alternative program architectures that do not rely on accepting incoming connections from the Internet (and hence do not need port mappings at all); these architectures usually require a third machine somewhere on the Internet (such as a rendezvous server or a relay server), but they eliminate the need for incoming connections, and operate just fine with outgoing connections only.

But "security through obscurity" is never the answer. UPnP is here. Here's how to use it.

go back to top

The demo program

The demo program is conceptually broken into two parts: an engine for the discovery of port mappings and for changing them, and a UI that uses the engine to allow the user to do what he wants. The source code itself is set up in a VC++ 6.0 workspace with four different configurations: Unicode and non-Unicode, debug and release for both.

Microsoft's implementation of UPnP relies on COM, so you might need to become familiar with COM-style types such as BSTR and VARIANT. The source code makes liberal use of ATL macros such as T2OLE for conversion, where needed.

go back to top

The engine

The engine actually performs three distinct types of tasks: device discovery (i.e., finding the router and getting the device information about it), retrieval of and changes to port mappings, and change-event notifications. Here are the public methods:

// simplified view of PortForwardEngine.h


class CPortForwardEngine  
{
public:

    CPortForwardEngine();
    virtual ~CPortForwardEngine();
    
    HRESULT ListenForUpnpChanges(
        CPortForwardChangeCallbacks *pCallbacks = NULL);
    HRESULT StopListeningForUpnpChanges( );

    BOOL GetDeviceInformationUsingThread( HWND hWnd ); 
    BOOL GetMappingsUsingThread( HWND hWnd ); 
    BOOL EditMappingUsingThread( PortMappingContainer& oldMapping, 
                       PortMappingContainer& newMapping, HWND hWnd ); 
    BOOL AddMappingUsingThread( PortMappingContainer& newMapping, 
                                                         HWND hWnd );
    BOOL DeleteMappingUsingThread( PortMappingContainer& oldMapping, 
                                                         HWND hWnd ); 
    
    std::vector<PortMappingContainer> GetPortMappingVector() const;  
    DeviceInformationContainer GetDeviceInformationContainer() const;
    
    BOOL IsAnyThreadRunning() const;  
};

The first thing you might notice is the reliance on threads. As implemented by Microsoft, UPnP relies on COM, and in this instance, COM is slow, usually requiring around three (3) seconds to complete, and sometimes requiring as many as ten (10) seconds. Not all the COM-related UPnP methods block during the time that they execute, but many do. The engine is therefore multi-threaded so that calls to its methods will not block your UI.

The threads created by the engine post notification messages to your UI to advise your application of progress through the thread's execution, and to advise you when the thread is complete. This is the reason why the threaded functions each take a HWND as a parameter; this parameter is the window to which messages are posted. The same message is used for all of the engine's functions; the message is a UINT named UWM_PORT_FORWARD_ENGINE_THREAD_NOTIFICATION. The meaning of the message is encoded in the WPARAM and LPARAM values of the message. The actual value of the message is obtained by a call to ::RegisterWindowMessage(), so your UI must be prepared to handle registered messages. For MFC users, this means that your message map will use the ON_REGISTERED_MESSAGE() macro.

The IsAnyThreadRunning() function is provided so that you can test whether any thread is running, before the shut-down of the program. This allows the thread object to delete itself, and prevents unintended memory leaks of CWinThread objects.

For device discovery, your application should call the GetDeviceInformationUsingThread() function, which tries to find a UPnP-enabled router on the local network and to obtain information about the router (such as model name, manufacturer etc.) if a router is found. When the thread is finished (as signified by the thread's posting of a UWM_PORT_FORWARD_ENGINE_THREAD_NOTIFICATION message), the UI can call GetDeviceInformationContainer() to get a structure containing information about the device. Device discovery is based on the COM interface IUPnPDeviceFinder, which is part of Microsoft's "Control Point API". Documentation can be found at MSDN entitled "Control Point API Reference".

Here's an explanation of the inner workings of GetDeviceInformationUsingThread(). Inside the thread created by the GetDeviceInformationUsingThread() function, CoCreateInstance() is called to get an instance of IUPnPDeviceFinder, and IUPnPDeviceFinder::FindByType() is called to get a IUPnPDevices collection of devices that match the requested type of device. The collection is enumerated/traversed to find each individual IUPnPDevice interface, and the following functions are called on the IUPnPDevice interface (see the MSDN documentation for IUPnPDevice):

IUPnPDevice::IconURL()
IUPnPDevice::get_Property()
IUPnPDevice::get_Children()
IUPnPDevice::get_Description()
IUPnPDevice::get_FriendlyName()
IUPnPDevice::get_HasChildren()
IUPnPDevice::get_IsRootDevice()
IUPnPDevice::get_ManufacturerName()
IUPnPDevice::get_ManufacturerURL()
IUPnPDevice::get_ModelNumber()
IUPnPDevice::get_ModelName()
IUPnPDevice::get_ModelNumber()
IUPnPDevice::get_ModelURL()
IUPnPDevice::get_ParentDevice()
IUPnPDevice::get_PresentationURL()
IUPnPDevice::get_RootDevice()
IUPnPDevice::get_SerialNumber()
IUPnPDevice::get_Services()
IUPnPDevice::get_Type Uniform()
IUPnPDevice::get_UniqueDeviceName()
IUPnPDevice::get_UPC()

Frankly speaking, once the IUPnPDevices collection is obtained, the code is a bit tedious. The code is based in part on the sample code found at MSDN: "Device Collections Returned by Synchronous Searches".

For retrieval of mappings, your application should call the GetMappingsUsingThread() function. When the function's thread completes (again, as signified by the thread's posting of a UWM_PORT_FORWARD_ENGINE_THREAD_NOTIFICATION message to your UI), the UI can call GetPortMappingVector() to get a std::vector which contains a collection of structures, each containing information about one mapping. Your application can also make changes to the mappings using the three self-explanatory functions of EditMappingUsingThread(), AddMappingUsingThread(), and DeleteMappingUsingThread().

The GetMappingsUsingThread() function is based on the COM interface IUPnPNAT, which is part of Microsoft's "NAT Traversal API". Documentation on this interface is scarce. For some reason, Microsoft has chosen to group this API with its API for "Internet Connection Sharing and Internet Connection Firewall". In addition to the difficulties caused by grouping NAT traversal with connection sharing, the on-line MSDN documentation for the NAT Traversal API does not have a separate entry in the table of contents, and does not sync well. The base page for documentation on IUPnPNAT is found at MSDN, entitled (simply) "IUPnPNAT".

Inside the thread created by the GetMappingsUsingThread() function, CoCreateInstance() is called to get an instance of IUPnPNAT, and IUPnPNAT::get_StaticPortMappingCollection() is called to get a IStaticPortMappingCollection collection of static port mappings. The collection is enumerated/traversed to find each individual IStaticPortMapping interface, and the following functions are called on the IStaticPortMapping interface (see the MSDN documentation for IStaticPortMapping):

IStaticPortMapping::get_ExternalIPAddress()
IStaticPortMapping::get_ExternalPort()
IStaticPortMapping::get_InternalPort()
IStaticPortMapping::get_Protocol()
IStaticPortMapping::get_InternalClient()
IStaticPortMapping::get_Enabled()
IStaticPortMapping::get_Description()

Much the same processing is performed inside the threads created by the other port-mapping functions (i.e., inside the threads created by EditMappingUsingThread(), AddMappingUsingThread(), and DeleteMappingUsingThread()), except that different ones of the IStaticPortMapping functions are called, as follows:

IStaticPortMapping::Enable()
IStaticPortMapping::EditDescription()
IStaticPortMapping::EditInternalPort()
IStaticPortMapping::EditInternalClient()

Event notification is interesting: every time there is a change in your router's configuration, it broadcasts (UDP) the change over the network. Microsoft's COM interface to UPnP can be configured to listen for these broadcasts, and to call callbacks within your program (if you register the callbacks properly). The changes are most commonly a change in a port mapping, but a notification is also received when there is a change in the router's external IP address.

Implementation of event notification requires an actual implementation of all the virtual functions for the two COM interfaces of INATExternalIPAddressCallback and INATNumberOfEntriesCallback. Then, an interface to the IUPnPNAT's event manager (INATEventManager) is obtained through a call to IUPnPNAT::get_NATEventManager(). Using the INATEventManager interface, it's possible to register the implementation of the derived INATExternalIPAddressCallback interface (INATEventManager::put_ExternalIPAddressCallback()) and to register the implementation of the derived INATNumberOfEntriesCallback interface (INATEventManager::put_NumberOfEntriesCallback()).

Because INATExternalIPAddressCallback and INATNumberOfEntriesCallback are, in a sense, COM servers, it is necessary to run them in the same thread as your main program, and that's how they're implemented (in a single-threaded apartment ("STA") model). Thus, event notifications are not run in a separate thread, unlike all the other functions we have discussed so far.

To get these notifications in your application, call ListenForUpnpChanges(). The function takes a pointer to a CPortForwardChangeCallbacks object, but if you pass in NULL, the engine will use a default object. (CPortForwardChangeCallbacks is defined in the same source and header files as CPortForwardEngine, which is usually not recommended but which helps to keep these classes all in one place.) The default object simply displays a ::MessageBox() indicating that there has been a change. For more elaborate handling of change-notification-events, derive your own class from CPortForwardChangeCallbacks and override the virtual functions OnNewNumberOfEntries() and OnNewExternalIPAddress(). Here is the definition of the CPortForwardChangeCallbacks class:

class CPortForwardChangeCallbacks  
{
public:
    CPortForwardChangeCallbacks();
    virtual ~CPortForwardChangeCallbacks();
    
    virtual HRESULT OnNewNumberOfEntries( 
                        long lNewNumberOfEntries );
    virtual HRESULT OnNewExternalIPAddress( 
                        BSTR bstrNewExternalIPAddress );
    
};

To use your CPortForwardChangeCallbacks-derived class, new one of them on the heap and pass its pointer to the ListenForUpnpChanges() function, like so:

ListenForUpnpChanges( new CMyDerivedPortForwardChangeCallbacks() );

You do not need to keep track of the new'd pointer; the engine will automatically delete the object for you when it's finished with it.

go back to top

The UI

The UI uses the engine in fairly unsurprising ways. When the program is started, it immediately calls GetDeviceInformationUsingThread() to get and display information about any UPnP-enabled routers on the LAN. If a router is found, its name is displayed, and more information about it can be displayed by clicking on the "More information ..." button, which displays all the information obtained from the GetDeviceInformationUsingThread() function in the following dialog:

Below the list of port mappings are four buttons that allow the user to retrieve port mappings from the router and to edit/add/delete them. Clicking one of these buttons invokes a corresponding one of the thread functions GetMappingsUsingThread(), EditMappingUsingThread(), AddMappingUsingThread(), or DeleteMappingUsingThread(). In addition, hidden progress bar controls are shown, and other changes are made to the appearance of the UI. Here's an example of the dialog that you see when a port mapping is added; a similar dialog is displayed when a mapping is edited. Only a confirmation-style dialog is displayed when a mapping is deleted.

As mentioned above, the threads post messages to the UI to advise it of the thread's progress. Here's a simplified view of the message-handler in the UI, which responds to these messages:

static const int msgPortRetrieve = 
       0x00F0 & CPortForwardEngine::EnumPortRetrieveDone;
static const int msgDeviceInfo = 
       0x00F0 & CPortForwardEngine::EnumDeviceInfoDone;
static const int msgAddMapping = 
       0x00F0 & CPortForwardEngine::EnumAddMappingDone;
static const int msgEditMapping = 
       0x00F0 & CPortForwardEngine::EnumEditMappingDone;
static const int msgDeleteMapping = 
       0x00F0 & CPortForwardEngine::EnumDeleteMappingDone;

afx_msg LRESULT 
  CPortForwardView::OnMappingThreadNotificationMessage(
  WPARAM wParam, LPARAM lParam)
{
    switch ( wParam & 0x00F0 )
    {
    case msgPortRetrieve:
        
        if ( wParam == CPortForwardEngine::EnumPortRetrieveInterval )
        {
            // this is a periodic notification message;

            // update the progress control

            
            m_ctlProgressComUpdate.SetPos( lParam );
        }
        else if ( wParam == CPortForwardEngine::EnumPortRetrieveDone )
        {
            // the thread is finished

            
            if ( !SUCCEEDED(lParam) )
            {
                // error: display message and take other action

            }
            else
            {
                // finished with no error,

                // get the vector of mappings and use it

                
                std::vector<CPortForwardEngine::
                     PortMappingContainer> mappingContainer;
                mappingContainer = 
                  m_PortForwardEngine.GetPortMappingVector();
        
                // display the port mapping and otherwise use them, etc....

            }

            // restore the appearance of the UI

        }
        
        break;
        
    case msgDeviceInfo:
        
        if ( wParam == CPortForwardEngine::EnumDeviceInfoInterval )
        {
            // this is a periodic notification message;

            // update the progress control

            
            m_ctlProgressIgdDeviceInfo.SetPos( lParam );
        }
        else if ( wParam == CPortForwardEngine::EnumDeviceInfoDone )
        {
            if ( SUCCEEDED(lParam) )
            {
                // finished with no error;

                // get device information and use it


                m_DeviceInfoContainer =  
                  m_PortForwardEngine.GetDeviceInformationContainer( );

                // display the device information

                // and otherwise use it, etc...

            }
            else
            {
                // error: display message and take other action                

            }

            // restore the appearance of the UI


        }
        
        break;
        
    case msgAddMapping:
        
        if ( wParam == CPortForwardEngine::EnumAddMappingInterval )
        {
            // this is a periodic notification message;

            // update the progress control

            
            m_ctlProgressAddUpdate.SetPos( lParam );
        }
        else if ( wParam == CPortForwardEngine::EnumAddMappingDone )
        {
            // the thread is finished

            
            if ( !SUCCEEDED(lParam) )
            {
                // error: display message and take other action                

            }
            else
            {
                // finished with no error

            }

            // restore the appearance of the UI


        }
        
        break;
        
    case msgEditMapping:
        
        // ... same as above for msgAddMapping

        
        break;
        
    case msgDeleteMapping:
        
        // ... same as above for msgAddMapping

        
        break;
        
    default:
        ASSERT ( FALSE );  // should never get here

    }
    return 0L;
}

The enums are defined in the CPortForwardEngine class. Basically, there are five kinds of messages (as measured by the WPARAM value of the message): messages for device information, for port mapping retrieval, for editing a mapping, for deleting a mapping, and for adding a mapping. There's a separate section of a switch statement for each different type of message, and each section then interprets the value passed by the LPARAM of the message.

The precise values of the WPARAMs and the LPARAMs are explained in the comments in the source code for the engine.

Change-event notifications are selected by checking the box labeled "Automatically listen for changes in the router". When this box is checked, the UI calls the engine's ListenForUpnpChanges() function, providing the engine with a pointer to a CPortForwardChangeCallbacks-derived object. The derived class is only a bit more complex than the default class: it simply displays a message asking the user if he wants to update the list of port mappings automatically now or manually later. Here's an example of the notification dialog when a change is detected in the number of mappings; a similar dialog is also displayed if a change is detected in the external IP address of the router:

The UI also has a built-in web server, which is a nice feature for testing whether a port-forward mapping is operational. The user checks the box labeled "Start Web server", which in turn enables the web-server controls on the right-hand side of the display. The user can then test for incoming Internet connectivity, with the test results being displayed in the user's default browser.

To get this to work, the user must create a port mapping on the same port as the listening port (which defaults to the arbitrary value of "9542" in the program). Once the port mapping has been created, and the program is set to listen for incoming connections, the test will cause a connection to be made to an external web-based proxy, which in turn will connect back through the user's machine using the newly-created port mapping. A successful test will look like this:

If the test fails, it's possible that the external proxy is not working properly. The program is pre-configured with two proxies, so one of them should always be working. New proxies can be added (and non-working ones deleted) through modifications to the program's PortForward.ini file. Other changes can also be made to this ini file, which is largely self-explanatory.

go back to top

Installation package

The download includes an installation package that installs the PortForward executable along with a short "Help" file (PortForward.chm). The installation package is named "PortForward100-Setup.exe", and the only changes made to your machine are the creation of a new directory (usually "c:\\Program Files\PortForward") and installation there of the executable and a few support files. There are no other changes made to your machine, not in the registry, not in the Windows folder, not anywhere else. There's even an un-installation program, so you can install the program with confidence. This installation package was built using Inno Setup, which I highly recommend.

go back to top

A quick rant

Although this article is finished, I can't resist the opportunity for a quick rant.

Why is it so hard to find documentation about this stuff? COM is difficult enough as it is (at least for me) without those difficulties being compounded by documentation that's hard to find, poorly organized, and incomplete. Microsoft's documentation for the IUPnPNAT interface, for example, is terrible. Just learning that IUPnPNAT is the interface you need is hard (search MSDN for "NAT Traversal API" and you won't find it). Once you determine that IUPnPNAT is the right interface, you're faced with a documentation that's poorly organized (why is it grouped as part of the ICS/Firewall API?) and largely incomplete. As one example, you can use the IUPnPNAT interface to get a collection of port mappings called IStaticPortMappingCollection, from which you can get an enumerator for each individual mapping. Not terribly straightforward, but normal enough in the COM world. But the enumerator that's returned is actually an IUnknown, and you need to guess at which one of the ga-jillion different kinds of IEnumXXXXs it actually is (it's an IEnumVARIANT, in case you're interested, which might seem evident after the fact but it certainly was not clear to me beforehand).

And that's nothing compared to the pain of implementing callback interfaces for the INATEventManager.

Web searching doesn't help much either. At the time of writing (February 2006), a Google search for IUPnPNAT turned up exactly 36 hits. 36!! And most of those hits were from people looking for help.

Likewise, the documentation for IUPnPDeviceFinder could be much better. For example, to find a router, you need to give IUPnPDeviceFinder a BSTR that "specifies the type URI for the device". What's that? Searching at the UPnP.org website provided no help; there's nothing there that gives examples of the URI naming convention, or provides examples of the commonly-used URIs.

At the end of the day, the "type URI" for a router turns out to be "urn:schemas-upnp-org:device:InternetGatewayDevice:1". No kidding. Does that surprise you or in any way seem to you like it might be self-explanatory or intuitive?

Rant over. Thanks for listening.

go back to top

Bibliography and reference materials

Here, in one place, is a list of all the articles and links mentioned in the article, as well as a few extra articles that might be helpful. Clicking the link will open a new window.

General UPnP and NAT translation links:

• UPnP.org
• Overview of Network Address Translation (NAT) in Windows XP
• Ensuring Great Experiences with NAT Traversal and Universal Plug and Play in Windows XP

COM-related links:

• "Control Point API Reference", which is used for UPnP device discovery.
• "IUPnPDeviceFinder", which is the starting interface device discovery using UPnP.
• Sample code for traversing/enumerating through the device collection returned by IUPnPDeviceFinder::FindByType().
• "Network Address Translation Traversal Reference", also known aa the "NAT Traversal API".
• "IUPnPNAT", which is the starting interface for UPnP control over port mappings in a router.

Inno Setup:

• Inno Setup
• ISTools, a graphical front-end for creating Inno scripts.

go back to top

License information

The source code is licensed under the MIT X11-style open source license. Basically, under this license, you can use/modify the code for almost anything you want. For a comparison with the BSD-style license and the much more restrictive GNU GPL-style license, click here.

History

• 2^nd March, 2006
  • Original release of the code and this article.
• 12^th March, 2006
  • Updated the article to include a few more graphics and other changes. No change to the code.

go back to top