Click here to Skip to main content
Click here to Skip to main content
Go to top

Installer Testing and Verification

, 22 Jun 2009
Rate this:
Please Sign up or sign in to vote.
Installer testing verifier - Useful for installer testing and verification.

Introduction

The main purpose of iCheckEngine is to aid testers in installer testing. In general, during installer testing, a tester matches the test scenario provided in the Install Specification.

This includes test against:

  1. File(s) deployment - Test hundreds of files/folders if they are deployed, and for their file sizes and file content.
  2. Registry(s) - Test registry entries for their creation and value.
  3. Executable version - Test if the correct version of the executable is deployed.
  4. Database - Test pertaining to databases, i.e., Tables/Procedures/Triggers are created, and connectivity and password are correct.
  5. XML configuration - XML values in configuration are properly done.
  6. Settings for environment variables

Note: Above is an unexhausted list, there could be many more tests which need to done for an installer testing, inputs are welcome.

We felt the need for this utility when we had many installers to test. Therefore, I worked to make some part of it automatic.

iCheckEngine provides users with the components mentioned below to test scenarios related to installer testing. At the end of each run, it can be configured to create either a file or HTML based report.

I was able to identify the following components which are required for a simple installation testing (inputs are most welcome).

  • File - Files are properly deployed or not, their size, version, content, etc.
  • Registry - Registry keys are created, values are set.
  • XML - Many applications nowadays store their configuration in XML files, therefore checks related to XML nodes.
  • Database - Checks pertaining to DB, tables, procedures, and triggers.
  • String check - It is useful to test environment variables against some general string match.

Two kinds of report generation are supported - HTML and file based:

  • FileLog - Provides functions useful for file based report generation.
  • HtmlLog - Provides functions useful for HTML based report generation.

Understanding iCheckEngine

The iCheckEngine application can be divided into two parts:

  1. iCheckEngine server and parser: The iCheckEngine server and parser are integrated into the main iCheckEngine.exe. Components consist of a number of DLLs which expose a Check function to be called by iCheckEngine.exe.
  2. iCheckEngine components: Components can be further divided into two parts:
    1. Internal - which are integrated into the main iCheckEngine.exe
    2. External - They expose check functions from the DLL
    3. Note: External components need to be explicitly declared in the file iCheckComponentConfig.xml with tmethods they expose. To keep things simple, these exported methods have a fixed signature as defined by the function pointer ISimpleFunc.

Given below are four important configuration files needed for iCheckEngine:

  1. iCheckConfig.xml - Main configuration files to be provided from command line arguments to iCheckEngine.
  2. Components file - Path for these files are mentioned in iCheckConfig.xml.
  3. Install check file - Path for these files is also mentioned in iCheckConfig.xml.
  4. Most important: It is possible to mention multiple InstallCheck nodes. All files will be processed and reports will be generated based on individual file configurations.

  5. Log file - File path application activity will be logged during parsing/execution of iCheckConfig.

Shown below is the sample content of iCheckConfig.xml:

<Install-Check version = "1.0.0.1">
    <Components FilePath="C:\iCheckEngine\config\iCheckComponentConfig.xml" />
    <LogFilePath FilePath="C:\iCheckEngine\Logs.txt" />    
    <InstallCheck FilePath="C:\iCheckEngine\config\iCheckSampleTest.xml" />
    <InstallCheck FilePath="C:\iCheckEngine\config\iCheckSampleTest_2.xml" />
</Install-Check>

iCheckComponentConfig.xml contains the definition for external components and method they expose. To add new components, these files need to be modified appropriately.

Below is a sample content of the file iCheckComponentConfig.xml:

<?xml version="1.0" encoding="UTF-8"?> 
<Install-Check>

<Components>
    <File Enabled="TRUE" Path="C:\iCheckEngine\iFileComponent.dll">
        <Methods>                
            <FileExistenceCheck />
            <FileNotExistenceCheck />
            <DirExistenceCheck />
            <DirNotExistenceCheck />
            <FileSizeGreater />    
            <FileSizeLesser />
            <CheckFileVersion />
            <MatchFileContent />
        </Methods>
    </File>

    <Registry Enabled="TRUE" Path="C:\iCheckEngine\iRegistryComponent.dll">
        <Methods>
            <KeyExistenceCheck />
            <KeyNotExistenceCheck />
            <KeyValueCheck />
        </Methods>
    </Registry>            
    <FileLog Enabled="FALSE" Path="C:\iCheckEngine\iFileLog.dll">
        <Methods>
            <WriteReportHeader />
            <WriteReportTrailer />
            <WriteMessage /> 
        </Methods>
    </FileLog>
    <HTMLLog  Enabled="TRUE" Path="C:\iCheckEngine\iFileLog.dll">
        <Methods>
            <WriteHTMLReportHeader />
            <WriteHTMLReportTrailer />
            <WriteHTMLMessage /> 
        </Methods>
    </HTMLLog>

        .....Content Stripped.................

Section in iCheckSampleTest.xml:

  • Variable - User defined variables (Note: Environment variables are automatically loaded).
  • ExecuteAtStart - Component method to be called at start of check execution.
  • ExecuteForEachCheck - Component method to be called at call of every check function.
  • ExecuteAtEnd - Component method to be called at end of check execution.
  • Alerts - Alerts need to be defined here.

iCheckSampleTest.xml holds actual instructions to call the appropriate install component functions for designated tasks. Below is the sample content of a file which contains instructions for an installer check: iCheckSampleTest.xml:

<Install-Check>
    <Variables>    
    <HTMLFilePath value ="C:\iCheckEngine\Reports\TestHtml.html" />
    <ReportFilePath value ="C:\iCheckEngine\Reports\TestReport.txt" />
    <ErrorFilePath value="C:\iCheckEngine\Reports\Data\Errors.txt" />
    <HTMLHeaderTemplate value="C:\iCheckEngine\ReportTemplates\
                    HTMLHeaderTemplate.txt" />
    <HTMLFootorTemplate value="C:\iCheckEngine\ReportTemplates\
                    HTMLFootorTemplate.txt" />
    <HTMLMessageTemplate value="C:\iCheckEngine\ReportTemplates\
                    HTMLMessageTemplate.txt" />

    <HeaderTemplate value=
        "C:\iCheckEngine\ReportTemplates\HeaderTemplate.txt" />
    <FootorTemplate value=
        "C:\iCheckEngine\ReportTemplates\FootorTemplate.txt" />
    <MessageTemplate value=
        "C:\iCheckEngine\ReportTemplates\MessageTemplate.txt" />
    </Variables>

    <ExecuteAtStart>    
    <!--This function will be called before 
            install-verifier start the check.-->
        
    <HTMLLog>
        <WriteHTMLReportHeader>
        <Parameter Filepath="@HTMLFilePath" HTMLHeaderTemplate="
        @HTMLHeaderTemplate" HTMLFootorTemplate="@HTMLFootorTemplate" 
        HTMLMessageTemplate="@HTMLMessageTemplate"/>
        </WriteHTMLReportHeader>        
    </HTMLLog>
    <FileLog>
        <WriteReportHeader>
         <Parameter  Filepath="@ReportFilePath" HeaderTemplate="
         @HeaderTemplate" FootorTemplate="@FootorTemplate" 
         MessageTemplate="@MessageTemplate"/>
        </WriteReportHeader>    
    </FileLog>
    </ExecuteAtStart>

    <ExecuteForEachCheck>
    <!--This function will be called for each checks and 
    every function it executes.-->
    <FileLog>
        <WriteMessage>
        <Parameter  Message="#OutputMessage" Status="#Status" 
         Check="#CheckName" Component="#Component" Method="#Method" />
        </WriteMessage>
    </FileLog>
    <HTMLLog>
        <WriteHTMLMessage>
         <Parameter  Message="#OutputMessage" Status="#Status" 
         Check="#CheckName" Component="#Component" Method="#Method" />
        </WriteHTMLMessage>
    </HTMLLog>
    </ExecuteForEachCheck>

    <Alerts>
    <Check Id="Test">            
        <StringCheck>
        <Equals>
             <Parameter  String1="@USERNAME" String2="
             bd98101" CaseSensitive="true" />
              <Parameter  String1="@USERNAME" String2="
              bd98101" CaseSensitive="false" />
        </Equals>
        <EqualsLike>
            <Parameter  String1="@OS" String2="windows" 
                    CaseSensitive="true"/>
        </EqualsLike>
        </StringCheck>
            
        <File>                 
        <FileExistenceCheck>
            <Parameter Filepath="C:\AUTOEXEC.BAT" />
            <Parameter Filepath = "C:\FileNotExist.txt" />
        </FileExistenceCheck>
        <CheckFileVersion>
            <Parameter Filepath = "C:\WINNT\TASKMAN.EXE" 
            Version = "5.0.2134.1"/>
        </CheckFileVersion>
        <MatchFileContent>
            <Parameter File1 = "C:\iCheckEngine\TempFiles\File1.txt" 
            File2 = "C:\iCheckEngine\TempFiles\File2.txt" 
            CaseSensitive="true"/>
        </MatchFileContent>
        </File>        
    </Check>

    <Check Id="CheckDirectoryExist">
        <File> 
        <DirExistenceCheck>
            <Parameter Dirpath="C:\Winnt"/>
            </DirExistenceCheck> 
            <DirExistenceCheck>
            <Parameter Dirpath="@windir\system32"/>
        </DirExistenceCheck>
        </File>
    </Check>            

    <Check Id="CheckFileSizes">
        <File> 
        <FileSizeGreater>
            <Parameter Filepath="C:\Test.txt" size="10240" />
        </FileSizeGreater>    
        <FileSizeLesser>
            <Parameter Filepath="C:\Test.txt" size="10240" />
        </FileSizeLesser>
        </File>
    </Check> 
        
    <Check Id="CheckRegistry">
        <Registry> 
        <KeyExistenceCheck>
                <Parameter KeyPath="SOFTWARE\Microsoft\
            Internet Explorer\W2kVersion" 
            KeyNode="HKEY_LOCAL_MACHINE"/>
        </KeyExistenceCheck>

        <KeyValueCheck>
            <Parameter ExpectedValue="6.0.2800.1106" 
              ValueName="W2kVersion"
            KeyPath="SOFTWARE\Microsoft\Internet Explorer" 
            KeyNode="HKEY_LOCAL_MACHINE"/>
        </KeyValueCheck> 
        </Registry>
    </Check> 
    </Alerts>
        
    <ExecuteAtEnd>
    <!--This function will be called after install-verifier completes the check.-->
    
        <HTMLLog>            
                <WriteHTMLReportTrailer>
           <Parameter Filepath="@HTMLFilePath" GeneratedDateTime="
           #CurrentDateTime" TotalCheck="#TotalCheck" SuccessCount="
           #SuccessCount" FailedCount="#FailedCount" 
        iVersion = "#iVersion"/>
              </WriteHTMLReportTrailer>
        </HTMLLog>        
        <FileLog>
        <WriteReportTrailer>
           <Parameter Filepath="@ReportFilePath" GeneratedDateTime="
           #CurrentDateTime" TotalCheck="#TotalCheck" SuccessCount="
           #SuccessCount" FailedCount="#FailedCount" iVersion = "#iVersion"/>
             </WriteReportTrailer>
        </FileLog>    
    </ExecuteAtEnd>
</Install-Check>

I believe "A picture speaks 1000 words", and for coding, the output speaks for sabout 1000 lines of code...

Sample HTML Output

Sample Text Output

iCheckText_Output.JPG

Thanks

Special thanks to Alex Hazanov for his XML Wrapper that helped in reading XML files.

History

  • Updates 22-June-2009:
    • Source download file:
      • Fixed compiler issue pertaining to Debug and Release versions.
      • Now uses generic paths to allow projects to be extracted in any folder and Run.
    • Article text:
      • Added HTML and TEXT output images to the article.
    s

License

This article, along with any associated source code and files, is licensed under The Code Project Open License (CPOL)

Share

About the Author

Kuldip Rindani
Systems Engineer
India India
C++, Unix Shell Scripts, DOTNET, ASP.NET, Sql Server 2000 and Oracle.

Comments and Discussions

 
General+ RemoteInstall PinmemberdB.19-Nov-09 2:14 
QuestionRelease version PinmemberLOUIS Christian17-Jun-09 4:15 
AnswerRe: Release version PinmemberKuldip Rindani17-Jun-09 4:22 
GeneralRe: Release version PinmemberLOUIS Christian18-Jun-09 2:23 
GeneralRe: Release version PinmemberLOUIS Christian18-Jun-09 4:00 
GeneralRe: Release version PinmemberKuldip Rindani18-Jun-09 4:44 
GeneralRe: Release version PinmemberLOUIS Christian18-Jun-09 20:33 
AnswerRe: Release version PinmemberKuldip Rindani17-Jun-09 5:00 
GeneralRe: Release version PinmemberLOUIS Christian18-Jun-09 1:02 
GeneralCrossplatform PinmemberSpatlabor15-Jun-09 17:42 
GeneralRe: Crossplatform PinmemberKuldip Rindani15-Jun-09 22:40 
GeneralCode Download Pinmembersam.hill11-Jun-09 16:21 
GeneralRe: Code Download PinmemberKuldip Rindani11-Jun-09 18:18 
GeneralRe: Code Download Pinmembersam.hill11-Jun-09 19:18 

General General    News News    Suggestion Suggestion    Question Question    Bug Bug    Answer Answer    Joke Joke    Rant Rant    Admin Admin   

Use Ctrl+Left/Right to switch messages, Ctrl+Up/Down to switch threads, Ctrl+Shift+Left/Right to switch pages.

| Advertise | Privacy | Mobile
Web01 | 2.8.140926.1 | Last Updated 22 Jun 2009
Article Copyright 2009 by Kuldip Rindani
Everything else Copyright © CodeProject, 1999-2014
Terms of Service
Layout: fixed | fluid