Redirecting an arbitrary Console's Input/Output

• Download source files - 3.56 Kb
• Download demo project - 43.1 Kb

Introduction

To redirect the input/output of a console application is interesting and useful. You can display the child's output in a window (just like Visual Studio's output window), or search some keywords in the output string to determine if the child process has completed its work successfully. An old, 'ugly' DOS program could become an useful component of your fancy Win32 GUI program.

My idea is to develop a simple, easy to use redirector class which can redirect an arbitrary console, and won't be affected by the behavior of the child process.

Background

The technique of redirecting the input/output of a console process is very sample: The CreateProcess() API through the STARTUPINFO structure enables us to redirect the standard handles of a child console based process. So we can set these handles to either a pipe handle, file handle, or any handle that we can read and write. The detail of this technique has been described clearly in MSDN: HOWTO: Spawn Console Processes with Redirected Standard Handles.

However, MSDN's sample code has two big problem. First, it assumes the child process will send output at first, then wait for input, then flush the output buffer and exit. If the child process doesn't behave like that, the parent process will be hung up. The reason of this is the ReadFile() function remains blocked untill the child process sends some output, or exits.

Second, It has problem to redirect a 16-bit console (including console based MS-DOS applications.) On Windows 9x, ReadFile remains blocked even after the child process has terminated; On Windows NT/XP, ReadFile always returns FALSE with error code set to ERROR_BROKEN_PIPE if the child process is a DOS application.

Solving the block problem of ReadFile

To prevent the parent process from being blocked by ReadFile, we can simply pass a file handle as stdout to the child process, then monitor this file. A more simple way is to call PeekNamedPipe() function before calling ReadFile(). The PeekNamedPipe function checks information about data in the pipe, then returns immediately. If there's no data available in the pipe, don't call ReadFile.

By calling PeekNamedPipe before ReadFile, we also solve the block problem of redirecting a 16-bit console on Windows 9x.

The class CRedirector creates pipes and launchs the child process at first. then creates a listener thread to monitor the output of the child process. This is the main loop of the listener thread:

    for (;;)
    {
        // redirect stdout till there's no more data.
        nRet = pRedir->RedirectStdout();
        if (nRet <= 0)
            break;

        // check if the child process has terminated.
        DWORD dwRc = ::WaitForMultipleObjects(
            2, aHandles, FALSE, pRedir->m_dwWaitTime);
        if (WAIT_OBJECT_0 == dwRc)      // the child process ended
        {
            ...
            break;
        }
        if (WAIT_OBJECT_0+1 == dwRc)    // m_hEvtStop was signalled, exit
        {
            ...
            break;
        }
    }

This is the main loop of the RedirectStdout() function:

    for (;;)
    {
        DWORD dwAvail = 0;
        if (!::PeekNamedPipe(m_hStdoutRead, NULL, 0, NULL,
            &dwAvail, NULL))    // error, the child process might ended
            break;

        if (!dwAvail)           // no data available, return
            return 1;

        char szOutput[256];
        DWORD dwRead = 0;
        if (!::ReadFile(m_hStdoutRead, szOutput, min(255, dwAvail),
            &dwRead, NULL) || !dwRead)  
                 // error, the child process might ended
            break;

        szOutput[dwRead] = 0;
        WriteStdOut(szOutput);          // display the output
    }

WriteStdOut is a virtual member function. It does nothing in CRedirector class. However it can be overrided to achieve our specific target, like I did in the demo project:

    int nSize = m_pWnd->GetWindowTextLength();  
             // m_pWnd points to a multiline Edit control
    m_pWnd->SetSel(nSize, nSize);
    m_pWnd->ReplaceSel(pszOutput);      
           // add the message to the end of Edit control

To redirect DOS console based applications on NT/2000/XP

MSDN's solution is to launch an intermediate Win32 Console application as a stub process between the Win32 parent and the 16-bit console based child. In fact the DOS prompt program (on NT/XP it's cmd.exe, on 9x it's command.com) is a natural stub process we just need. We can test this in RedirDemo.exe:

1. Input 'cmd.exe' in Command Editbox, then press Run button.
2. Input the name of the 16-bit console based application (dosapp.exe for example) in the Input Editbox, then press Input button. Now we can see the output of the 16-bit consol.
3. Input 'exit' in the Input Editbox, then press Input button to terminate cmd.exe

Apparently this is not a good solution because it's too complicated. A more effective way is to use a batch file as the stub. Edit stub.bat file like this:

%1 %2 %3 %4 %5 %6 %7 %8 %9

Then run a command like 'stub.bat dosapp.exe', then the 16-bit DOS console application runs OK.