CLI Performance Testing

• Download source code - 5 KB

Introduction

I searched multiple places to get some knowledge on Command Line Interface (CLI) Performance Testing using Java, but the problem was I could not find any constructive step-by-step procedure. This made me think of jotting down the processes which would help others in future.

I have already implemented the same using CURL (you can use any CLI) which was executed in LoadRunner for Scalability, Reliability and various other performance testing.

Prerequisite

• It is expected that you know a bit of development in Java. You may not be an awesome developer but should know a bit more than "Hello World!" program.
• You should have knowledge in performance testing tool e.g. LoadRunner.
• You should have any Java Development IDE, e.g., NetBeans.

Requirement

• In this example I have used CURL but you can use any CLI application for performance testing. To know more about CURL, please visit cURL and libcurl
• Before you use any CLI application, please go through its documentation and various arguments it takes to accomplish a task.
• You require performance testing tool and in this example we will use LoadRunner. It is not limited to LoadRunner and could be used in performance testing tool.
• You require Java Development IDE and we will use NetBeans in this example. This is not required but will help us to write error free code and later copy paste in performance testing tool. When you write the java code in NetBeans, you get few advantages over writing the same in LoadRunner.
  1. Add the API in NetBean project easily.
  2. Required packages are imported in NetBeans automatically.
  3. Debug the code in hassle-free method.
  4. Make the code error-free with any fuss. 

Note: Before you start writing the Java Code make sure that whatever you write in NetBeans have to be copied and it has to be understood by LoadRunner.

Performance Scenario

Suppose you have an CLI application (e.g. CURL.exe) which was provided to you by development team of your project and you are responsible to provide performance metrics for that CLI when it is tested with 100 users concurrently running for 2 hour with a ThinkTime of 10 sec.

Transaction Details

• Login to Remote Server using the CURL
• Upload file from your machine to remote server using CURL
• Download file from remote server to your machine using CURL

Start writing the code in NetBeans

• In NetBeans, click File -> New Projects.
• Select Java -> Java Application and click Next.
• Enter Project Name as "Sumit_Cli_Example", select Project Location, select Create Main Class checkbox and select Finish.
• New Project will be created.

Note

The CLI (in this case CURL.exe) or the code below will not match your performance test script. This blog will just help you to understand how to execute Performance Test for CLI using Java. You will obviously have different CLI which will have different arguments and properties with which you have to write Java Code to execute in LoadRunner.

The best way to understand your CLI is to go through its documentation which should be provided by the development team at the time of delivering CLI.

package sumit_cli_example;
 
import java.io.BufferedReader;
import java.io.InputStreamReader;
 
public class Sumit_Cli_Example
{
    static Runtime objRuntime = null;
    static Process objProcess = null;
    static String strCommand = null;
    static boolean boolStatus = false;
    static BufferedReader objInputStream = null;
    static BufferedReader objErrorStream = null;
 
    static String strInputStream = "";
    static String strErrorStream = "";
    static String strXAuthToken = "";
    static String strCurlPath = "D:\\curl\\";
 
    public static void main(String[] args)
    {
        try
        {
            objRuntime = Runtime.getRuntime();
 
            /*Authentication*/
            strCommand = strCurlPath + "curl.exe -v -H \"X-Storage-User: system1:" + 
              "root1\" -H \"X-Storage-Pass: password1\" http://xxxxxx.com:7777/auth/v1.0";
 
            objProcess = objRuntime.exec(strCommand);
 
            strInputStream = "";
            boolStatus = false;
            objInputStream = new BufferedReader(new InputStreamReader(objProcess.getInputStream()));
            while (strInputStream != null)
            {
                strInputStream = objInputStream.readLine();
 
                if(strInputStream != null)
                {
                    System.out.println(strInputStream);
                    if(strInputStream.contains("X-Auth-Token") == true)
                    {
                        boolStatus = true;
                        strXAuthToken = strInputStream.replace("< X-Auth-Token: ", "");
                        System.out.println("X-Auth-Token:" + strXAuthToken);
                    }
                }
            }
 
            /*Above transaction has failed if boolStatus is false*/
            if(boolStatus == false)
            {
                System.out.println("X-Auth-Token was not found");
            }
            else
            {
                System.out.println("Success");
            }
           
            /*Uploading file*/
            strCommand = strCurlPath + "curl.exe -X PUT -T D:\\curl\\sumitbiswas.txt" + 
              " -H \"X-Auth-Token: " + strXAuthToken + 
              "\" http://xxxxxx.com:8080/v1/AUTH_admin/Container_1/";
 
            objProcess = objRuntime.exec(strCommand);
 
            strInputStream = "";
            boolStatus = false;
            objInputStream = new BufferedReader(new InputStreamReader(objProcess.getInputStream()));
            while (strInputStream != null)
            {
                strInputStream = objInputStream.readLine();
 
                if(strInputStream != null)
                {
                    System.out.println(strInputStream);
                    if(strInputStream.contains("201 Created") == true)
                    {
                        boolStatus = true;
                    }
                }
            }
 
            /*Above transaction has failed if boolStatus is false*/
            if(boolStatus == false)
            {
                System.out.println("Uploading file was not successful");
            }
            else
            {
                System.out.println("Success");
            }
           
            /*Downloading file*/
            strCommand = strCurlPath + "curl.exe -o D:\\curl\\sumitbiswas.txtxyz -H \"X-Auth-Token: " + 
              strXAuthToken + "\" http://xxxxxx.com:8080/v1/AUTH_admin/Container_1/sumitbiswas.txt";
 
            objProcess = objRuntime.exec(strCommand);
 
            strInputStream = "";
            boolStatus = false;
            objInputStream = new BufferedReader(new InputStreamReader(objProcess.getInputStream()));
            while (strInputStream != null)
            {
                strInputStream = objInputStream.readLine();
 
                if(strInputStream != null)
                {
                    System.out.println(strInputStream);
                    if(strInputStream.contains("201 sent") == true)
                    {
                        boolStatus = true;
                    }
                }
            }
 
            /*Above transaction has failed if boolStatus is false*/
            if(boolStatus == false)
            {
                System.out.println("Downloading file was not successful");
            }
            else
            {
                System.out.println("Success");
            }
        }
        catch(Exception ex)
        {
            System.out.println(ex);
        }
    }
}

Run the project and it should run successfully. It should connect to the Remote Server, Upload File and then Download File. Once you are satisfied with the output and desired result, its time to copy paste in LoadRunner.

Note:

You can use .getErrorStream() instead of .getInputStream() to get the error output stream and read it similar way as demonstrated in the above code.

Copy Paste above code in LoadRunner

In this section we will copy paste the working code from NetBeans in LoadRunner using Java Vuser protocol. We will then modify the code which will be recognizable by LoadRunner. We will also add parameters and other features so that we can load test the CLI for 100 users which will be running for 2 hours concurrently.

import lrapi.lr;
 
import java.io.BufferedReader;
import java.io.InputStreamReader;
 
public class Actions
{
    Runtime objRuntime = null;
    Process objProcess = null;
    String strCommand = null;
    boolean boolStatus = false;
    BufferedReader objInputStream = null;
    BufferedReader objErrorStream = null;
 
    String strInputStream = "";
    String strErrorStream = "";
    String strXAuthToken = "";
    String strCurlPath = "D:\\curl\\";
 
    public int init() throws Throwable
    {
    try
    {
        lr.start_transaction("CLI_01_Login");
 
            objRuntime = Runtime.getRuntime();
 
            /*Authentication*/
            strCommand = strCurlPath + "curl.exe -v -H \"X-Storage-User: " + 
              "<UserNames>\" -H \"X-Storage-Pass: password1\" http://xxxxxx.com:7777/auth/v1.0";
 
            objProcess = objRuntime.exec(strCommand);
 
            strInputStream = "";
            boolStatus = false;
            objInputStream = new BufferedReader(new InputStreamReader(objProcess.getInputStream()));
            while (strInputStream != null)
            {
                strInputStream = objInputStream.readLine();
 
                if(strInputStream != null)
                {
                    System.out.println(strInputStream);
                    if(strInputStream.contains("X-Auth-Token") == true)
                    {
                        boolStatus = true;
                        strXAuthToken = strInputStream.replace("< X-Auth-Token: ", "");
                        System.out.println("X-Auth-Token:" + strXAuthToken);
                    }
                }
            }
 
            /*Above transaction has failed if boolStatus is false*/
            if(boolStatus == false)
            {
        lr.error_message("X-Auth-Token was not found");
 
        lr.end_transaction("CLI_01_Login", lr.FAIL);
            }
            else
            {
        lr.message("Success");
 
        lr.end_transaction("CLI_01_Login", lr.PASS);
            }
 
        lr.think_time(10);
    }
        catch(Exception ex)
        {
            System.out.println(ex);
        }
    return 0;
    }
 

    public int action() throws Throwable
    {
    try
    {
        lr.start_transaction("CLI_02_UploadFile");
 
        /*Uploading file*/
            strCommand = strCurlPath + "curl.exe -X PUT -T D:\\curl\\sumitbiswas.txt" + 
              " -H \"X-Auth-Token: " + strXAuthToken + 
              "\" http://xxxxxx.com:8080/v1/AUTH_admin/Container_1/";
 
            objProcess = objRuntime.exec(strCommand);
 
            strInputStream = "";
            boolStatus = false;
            objInputStream = new BufferedReader(new InputStreamReader(objProcess.getInputStream()));
            while (strInputStream != null)
            {
                strInputStream = objInputStream.readLine();
 
                if(strInputStream != null)
                {
                    if(strInputStream.contains("201 Created") == true)
                    {
                        boolStatus = true;
                    }
                }
            }
 
            /*Above transaction has failed if boolStatus is false*/
            if(boolStatus == false)
            {
                lr.error_message("Uploading file was not successful");
 
        lr.end_transaction("CLI_02_UploadFile", lr.FAIL);
            }
            else
            {
                lr.message("Success");
 
        lr.end_transaction("CLI_02_UploadFile", lr.PASS);
            }
 
        lr.think_time(10);
 
        lr.start_transaction("CLI_03_DownloadFile");
           
            /*Downloading file*/
            strCommand = strCurlPath + "curl.exe -o D:\\curl\\sumitbiswas.txtxyz " + 
               "-H \"X-Auth-Token: " + strXAuthToken + 
               "\" http://xxxxxx.com:8080/v1/AUTH_admin/Container_1/sumitbiswas.txt";
 
            objProcess = objRuntime.exec(strCommand);
 
            strInputStream = "";
            boolStatus = false;
            objInputStream = new BufferedReader(new InputStreamReader(objProcess.getInputStream()));
            while (strInputStream != null)
            {
                strInputStream = objInputStream.readLine();
 
                if(strInputStream != null)
                {
                    if(strInputStream.contains("201 sent") == true)
                    {
                        boolStatus = true;
                    }
                }
            }
 
            /*Above transaction has failed if boolStatus is false*/
            if(boolStatus == false)
            {
                lr.error_message("Downloading file was not successful");
 
        lr.end_transaction("CLI_03_DownloadFile", lr.FAIL);
            }
            else
            {
                lr.message("Success");
 
        lr.end_transaction("CLI_03_DownloadFile", lr.PASS);
            }
 
        lr.think_time(10);
    }
        catch(Exception ex)
        {
            System.out.println(ex);
        }
    return 0;
    }
 

    public int end() throws Throwable
    {
    return 0;
    }
}

Note: Check that static keyword has been removed in LoadRunner because it is not called from static main method. Transaction Name has been defined. Parameter has been provided for UserNames. Normally the parameter is separated by {}, but here it has been set as <>. This can be configured from VuGen in Tools -> General Options -> Parameterization. Think Time has been provided and last but not the least transaction has been categorized as init(), action(), and end().

The curl path or the file path is with respect to the box where the script is running. Please change it accordingly with respect to the load generator machine. Run the LoadRunner script for 2 iterations and check whether it is successfully doing the operations. With this our LoadRunner script is ready and to be executed in LoadRunner controller as per the scenario defined.

Points of Interest

Make sure when you use Windows box, mark the string as double quote ( " ) and when using Linux box, mark the string as single quote ( ' ).

History

This is the first version.