URL Sniffer based on NetFilter SDK

• Download source - 22.37 KB

Introduction

In this little article, we are going to discuss one of various potential usages of NetFilter SDK that is a powerful framework for transparent filtering of Data Packets Exchanged, on Windows. By using NetFilter, we are able to develop Basic Firewalls, Network Monitors and all possible applications that have something to do with Data Packet Management.

In our application, we will use a higher abstraction level of raw data packet filtering, called Protocol Filtering. NetFilter is capable of advanced packet parsing, so we can work directly with protocols like HTTP, SMTP, POP3, SSL.

In our application, we will filter all outgoing HTTP Requests and dump the URL.

Using the Code

Before starting to code the URL Sniffer, we have to understand the architecture of NetFilter SDK. We have essentially two components, one at Kernel Mode (the filter driver) and a correspondant User Mode Interface, given by a DLL.

Filtering functionality is basically accomplished by netfilter2.sys, a Transparent Filter Driver, that hooks at NDIS Level and allows transparent operations, this means that we will not have any conflict with products like AntiVirus or Firewalls.

Our applications will not deal directly with the Kernel Mode component, but more easily we will interact with a couple of DLLs that offer us a complete set of APIs. The choice of DLL to use is truly easy, if you have to perform a ProtocolFiltering Application, the DLL to use is ProtocolFilters.dll, else nfapi.dll. You also have to include the involved libraries by using #pragma statement.

#pragma comment(lib,"nfapi.lib")
#pragma comment(lib,"ProtocolFilters")

Here an abstract from NetFilter's Help that will help you to clarify how the Filtering Architecture is structured:

• Implement the event handlers from PFEvents or PFEventsDefault. Include ProtocolFilters\include\ProtocolFilters.h to use C++ API.
• Initialize the library with pf_init() call. Specify event handler functions and a folder for storing library files.
• If the library is used for filtering data hooked by NetFilterSDK driver, initialize nfapi library by passing to nf_init() a pointer to ProtocolFilters internal event handlers returned from pf_getNFEventHandler() function. Add the filtering rules.
• Build a chain of filters for each filtered connection in tcpConnected event handler. It is also possible to add filters later if needed. A filter deletes itself from a chain in case it detects a different protocol. So it is possible to build the same chain from available filters for all connections.
• Filter the classified objects in dataAvailable/dataPartAvailable events.
• To stop filtering, call nf_free() first to stop indicating events from driver, then call pf_free() to uninitialize the internal structures in ProtocolFilters.

PFEvents handlers are executed in a context of a thread that calls input functions from pf_getNFEventHandler(). Nfapi indicates events from own separate thread, so it is necessary to synchronize access to variables used from event handlers and shared with other threads in your application, including the main thread.

Essentially the URL Filter needs to catch all Outgoing HTTP Requests and carve out from the packet the string that belong to the URL.

The first operation is to develop a class that is able to manage Packet Filtering Events:

class DumpFilter : public PFEventsDefault
{
public:
        DumpFilter()
        {
        }

        virtual void tcpConnected(nfapi::ENDPOINT_ID id, 
				nfapi::PNF_TCP_CONN_INFO pConnInfo)
        {
                if (pConnInfo->direction == NF_D_OUT)
                {
                        pf_addFilter(id, FT_HTTP, FF_READ_ONLY_OUT | FF_READ_ONLY_IN);
                }
        }

        void dataAvailable(nfapi::ENDPOINT_ID id, PFObject * object)
        {
                if (object->getType() == OT_HTTP_REQUEST)
                {
                        HttpUserActivity(object);
                }                       
        }
};

Let's explain a bit the member functions of this class. Basically this class belongs to PFEventsDefault EventHandler. In our application, we have basically to manage the kind of connection that we want to filter. This task is accomplished by tcpConnected() function where we add or remove the Protocol that we want to filter by using pf_addFilter(). In our case, we have to filter only HTTP Requests so the constant value to use is given by FT_HTTP.

At this point, the specific event to catch is defined, now we have to establish what to do when the Data of the Specified Event is available, for this reason we have dataAvailable(), inside we can make a primary packet parsing, in our case the Rule is that we want only HTTP requests, so object type is OT_HTTP_REQUEST. Now that we have the wanted activity, we can proceed with specific parsing in our case when HTTP Activity is revealed is called HttpUserActivity()

void HttpUserActivity(PFObject * object) // Carves the accessed URL
{
                PFHeader h;
                PFStream * pStream;
                std::string url;
                std::string status;

                if (object->getType() != OT_HTTP_REQUEST)
                        return ;

                if (pf_readHeader(object->getStream(HS_HEADER), &h))
                {
                        PFHeaderField * pField = h.findFirstField("Host");
                        if (pField)
                        {
                                url = "http://" + pField->value();
                        }
                }

                FILE *f = fopen("UrlDump.txt","a+");

                fprintf(f,"-> %s\n",url.c_str());
                fclose(f);

                return;         
}

You can see here, how it is easy to manage and parse a packet. What we want is to carve out the URL from the intercepted HTTP Request, at this point of the code we already have the packet, so we have only to search inside it the field that belongs to the URL. Our URL is given by the value of Host Field. SDK has an handy findFirstField(), we are looking for "Host" field that as you can see is easily extracted and dumped into a file.

This is the core filtering and sniffing part of our UrlSniffer. Let's finally see how is building the main().

int main(void)
{
        NF_RULE rule;

        nf_adjustProcessPriviledges();

        printf("Press any key to stop...\n\n");

        DumpFilter f;

        if (!pf_init(&f, L"c:\\netfilter2"))
        {
                printf("Failed to initialize protocol filter");
                return -1;
        }

        // Initialize and Start Filtering
        if (nf_init(NFDRIVER_NAME, pf_getNFEventHandler()) != NF_STATUS_SUCCESS)
        {
                printf("Failed to connect to driver");
                return -1;
        }
        
        // Filter all TCP connections
        memset(&rule, 0, sizeof(rule));
        rule.direction = NF_D_OUT;
        rule.protocol = IPPROTO_TCP;
        rule.filteringFlag = NF_FILTER;
        nf_addRule(&rule, TRUE);

        getchar();

        nf_free();
        pf_free();

        return 0;
}

In the main, we have only to specify Rules for Filtering.

Points of Interest

This SDK is truly flexible, once you have clear the general architecture, it will be matter of an hour or less to develop applications. Due to the NDIS hooking level, we are free from choosing an Interface Adapter. I want also to specify that the code of UrlSniffer is obtained by modifying PFHttpContentFilter.

History

• April 2010 - Alpha version