JMeter to check files and file content

Recently, we needed a JMeter which could perform some tests to make sure files were created on the file system,  matching a specific file path and filename, and some text had to match in the file itself. These files were created by a simulator, but to test our project, we wanted to automatically check if the simulator received all required files.

Apparently, JMeter is not often used for such a checks on the file system, but we preferred JMeter so we could combine the result reports nicely with the reports of the sending of the files (performed by another JMeter).

We succeeded by using BeanShell scripts in the JMeter with a BeanShell Sampler and a BeanShell Assertion. I created a template JMeter jmx and replaced some tags (@…@) in this template for each of our test cases. An example of a filled in JMeter jmx file based on this template is available here. We transform the template to the specific test case jmx file using Excel macros for easy configuration management.

In the JMeter we always linked a CheckFileSampler.bsh and CheckFileAssertion.bsh scripts to perform the required checks. Unlimited properties can be assigned in the JMeter jmx and these properties can very easily be retrieved inside the BeanShell script by using:

value = vars.get("<variable name>");

In the BeanShell script, one can just use standard Java code. I made a simplified version of the code, which is less related to our project. In this code, I build up the path of the file which is expected to exist. The file path is build up based on the properties set in the jmx file. I check if the file exists. If it could be found, I match some patterns against the content of the file. If all goes well, the test case succeeds, else it fails.

import java.util.ArrayList;
import java.util.Arrays;
import java.util.List;
import java.util.regex.Matcher;
import java.util.regex.Pattern;
import java.io.*;
import java.util.regex.*;
Failure=false;
FailureMessage="";
String testcase = vars.get("testcase");
String simulatorReceiveMessageBasePath = vars.get("simulatorReceiveMessageBasePath");
String partnerName = vars.get("partnerNameProperty");
//should the file exist (be received) or not for this partner? 
String partnerReceiveMessageRequired = vars.get(partnerName + "_partnerRequiredReceiveProperty");
//the following pattern should match within the file (if it exists), semicolon (;) separeted list
String patternsToMatchList = vars.get("patternsToMatchList");
//within the basepath, the simulator creates a new folder on every start of the simulator, we wanted to retrieve the newest folder
//retrieve newest folder from partner
File basefolder = new File(simulatorReceiveMessageBasePath);
if (!basefolder.isDirectory()) {
  Failure = true;
  log.error(testcase + " - Partner " + partnerName  + " incorrect base folder: " + simulatorReceiveMessageBasePath);
  FailureMessage = testcase + ";" + partnerName  + ";INCORRECT_BASE_FOLDER;" + simulatorReceiveMessageBasePath + "|" + FailureMessage;
  break;
}  
File[] foldersinbasefolder = basefolder.listFiles();
Comparator filecomp = new Comparator() {
      public int compare(Object o1, Object o2) {
        return new Long(((File)o2).lastModified()).compareTo
             (new Long(((File) o1).lastModified()));
      }
    };  
Arrays.sort( foldersinbasefolder, filecomp);    
if (foldersinbasefolder == null || foldersinbasefolder.length == 0) {
  Failure = true;
  log.error(testcase + " - Partner " + partnerName  + " no subfolder could be found in the base folder: " + simulatorReceiveMessageBasePath);
  FailureMessage = testcase + ";" + partnerName  + ";NOSUBFOLDERSINBASEFOLDER;" + simulatorReceiveMessageBasePath + "|" + FailureMessage;
  break;
}    
String partnerSimulatorFolder  = foldersinbasefolder[0].getCanonicalPath();
String partnerReceiveMessageFullPath = partnerSimulatorFolder + "\\" + testcase + "_message.xml";
if (partnerReceiveMessageRequired != null && partnerReceiveMessageRequired.equals("1"))  {
      
  //partner should have received the message
  log.info(testcase + " - Partner " + partnerName + " should have received message with full path " + partnerReceiveMessageFullPath);
  File f = new File(partnerReceiveMessageFullPath);
  if (f.exists() && f.isFile() && f.length() > 0) {
    log.info(testcase + " - Partner " + partnerName + " has received message with full path " + partnerReceiveMessageFullPath + " size: " + f.length() + "bytes");
      
    int patternsMatched = 0;
    if (patternsToMatchList != null && patternsToMatchList != "") {
      
      //check if the pattern can be matched with the file
      String[] patternsToMatch = patternsToMatchList.split(";");
      forloop:
        for (String patternToMatch : patternsToMatch) {
          boolean patternmatchresult = true;
          if(patternmatchresult == true) {
            Pattern regexp = Pattern.compile(".*" + patternToMatch + ".*");
            Matcher matcher = regexp.matcher("");
            LineNumberReader lineReader = null;
            try {
              lineReader = new LineNumberReader( new FileReader(f) );
              String line = null;
              boolean resultinlinewhileloop = false;
                while ((line = lineReader.readLine()) != null && resultinlinewhileloop) {
                  matcher.reset( line ); //reset the input
                  log.debug(testcase + " - Partner " + partnerName + ", line: " + line);
                  if ( matcher.find() ) {
                    patternsMatched++;
                    log.debug(testcase + " - Partner " + partnerName + ", match found in file for pattern: " + patternToMatch);
                    resultinlinewhileloop = true;
                    continue forloop; //continue with next pattern to match in file, doesn't seem to work
                  }
                }
                if(!resultinlinewhileloop) {
                  Failure=true;
                  patternmatchresult = false;
                  FailureMessage=testcase + ";" + partnerName + ";" + partnerReceiveMessageFullPath + ";NOPATTERNMATCH;" + patternToMatch + "|" +  FailureMessage;
                  log.error(testcase + " - Partner " + partnerName + ", pattern could not be matched against the file! " + patternToMatch);
                  break;
                }
            }
            catch (FileNotFoundException ex) {
              ex.printStackTrace();
            }
            catch (IOException ex){
              ex.printStackTrace();
            }
            finally {
              try {
                if (lineReader!= null) lineReader.close();
              }
              catch (IOException ex) {
                ex.printStackTrace();
              }
            }
          }
      }
      log.debug("patternsMatched:" + patternsMatched + ", patternsToMatch:" + patternsToMatch + ", length:" + patternsToMatch.length);
      //check if all patterns could be matched agains the file content
      if (patternsMatched == patternsToMatch.length) {
          log.info(testcase + " - Partner " + partnerName + ", all paterns could be matched against file. Test completed successfully!");
      }
      else {
        Failure=true;
        log.error(testcase + " - Partner " + partnerName + ", some patterns could not be matched against the file! " + patternsToMatchList.toString());
      }
    }
  }
  else {
    log.error(testcase + " - FILE NOT FOUND! Partner " + partnerName + " should have received message with full path " + partnerReceiveMessageFullPath);
    Failure=true;
    FailureMessage=testcase + ";" + partnerName + ";FILENOTFOUND;" + partnerReceiveMessageFullPath + "|" + FailureMessage;
  }
}

In our own project, we wanted to check different partners for every test case. Which resulted in this BeanShell.

We run our JMeter jmx files using Ant. This build file shows how I start our JMeter and create a nice looking report web page of it. I have some code to make some necessary transformations depending if I run the JMeters on our Windows or Linux machines.

<?xml version="1.0" encoding="UTF-8" ?> 
<project basedir="." default="dist_receive" name="Run JMeters to check files received">
  
  <taskdef resource="net/sf/antcontrib/antcontrib.properties"/>
  <property name="windows.local.basepath.location" value="R:" />
  <property name="linux.local.basepath.location" value="/home/user/" />
  
  <condition property="local.basepath.location"
    value="${windows.local.basepath.location}"
    else="${linux.local.basepath.location}">
    <os family="windows" />
  </condition>
  
  <condition property="target.os"
    value="windows"
    else="linux">
    <os family="windows" />
  </condition>  
  
  <property name="test.dir.fullpath" value="${local.basepath.location}/project/test"/>
  <property name="tools.dir.fullpath" value="${local.basepath.location}/tools"/>
  <property name="jmeter.dir.fullpath" value="${tools.dir.fullpath}/jakarta-jmeter-2.3.2"/>
  <property name="jmeter.install.dir.fullpath" value="${jmeter.dir.fullpath}"/>
  <property name="ant.dir.fullpath" value="${tools.dir.fullpath}/apache-ant-1.7.1"/>
  <property name="java.class.path" value="${tools.dir.fullpath}/jdk1.6.0_10/"/>
  <property name="ant.install.dir.fullpath" value="${tools.dir.fullpath}/apache-ant-1.7.1/bin"/>
  
  <property file="${jmeter.install.dir.fullpath}/bin/jmeter.properties"/>
  <property name="testcase.basepath" value="${test.dir.fullpath}/test_data"/>
  <property name="jmeter.result.file.basepath" value="${test.dir.fullpath}/test_results/"/>
  <property name="jmeter.result.filename" value="result"/>
  <property name="jmeter.result.extension" value=".xml"/>
  <property name="jmeter.report.filename" value="report"/>
  <property name="jmeter.report.extension" value=".html"/>
  <property name="jmeter.receive.filename" value="receive"/>
  
  <property name="jmeter.receive.result.file.fullpath" value="${jmeter.result.file.basepath}${jmeter.result.filename}_${jmeter.receive.filename}${jmeter.result.extension}"/>
  <property name="jmeter.receive.report.file.fullpath" value="${jmeter.result.file.basepath}${jmeter.report.filename}_${jmeter.receive.filename}${jmeter.report.extension}"/>
  
  <property name="testcase.receive.extension" value="*_receive.jmx"/>
  
  <taskdef name="jmeter" classname="org.programmerplanet.ant.taskdefs.jmeter.JMeterTask">
    <classpath>
      <pathelement location="${jmeter.dir.fullpath}/extras/ant-jmeter-1.0.9.jar"/>
    </classpath>
  </taskdef>
  
  <tstamp>
      <format property="current.date.time" pattern="MM/dd/yyyy hh:mm"/>
  </tstamp>
  <tstamp>
      <format property="xml.current.date" pattern="yyyy-MM-dd"/>
  </tstamp>
  
    
  <target name="dist_receive" description="run jmeter(s) to check received messages">
  
    <property name="test.cases.to.run.basepath" value="${testcase.basepath}/" />
    <property name="test.case.to.perform" value="" />  <!--if the props file does not contain a 'test.name' => the variable needs to be known but no processing should take place. Properties are immutable, if already set by prop file=> not overridden here! -->
    
    <delete file="${jmeter.receive.result.file.fullpath}" />
    <delete file="${jmeter.receive.report.file.fullpath}" />
  
    <if>
      <equals arg1="${test.case.to.perform}" arg2="" />
      <then>
        <echo message="All send cases in dir ${test.cases.to.run.basepath} will be run" />
        <antcall target="run_jmeters">
          <param name="loadtests.basepath" value="${test.cases.to.run.basepath}"/>
          <param name="loadtests.extension" value="${testcase.receive.extension}"/>
          <param name="jmeter.result.file.fullpath" value="${jmeter.receive.result.file.fullpath}"/>
        </antcall>
      </then>
      <else>
        <echo message="Only one testcase is run ${test.case.to.perform}" />
        <antcall target="run_jmeter">
          <param name="jmx.file.fullpath" value="${test.cases.to.run.basepath}/${test.case.to.perform}"/>
          <param name="jmeter.result.file.fullpath" value="${jmeter.receive.result.file.fullpath}"/>
        </antcall>
      </else>
    </if>
    
    <antcall target="make_report">
      <param name="jmeter.result.file.fullpath" value="${jmeter.receive.result.file.fullpath}"/>
      <param name="jmeter.report.file.fullpath" value="${jmeter.receive.report.file.fullpath}"/>
    </antcall>
  </target>
  
  <target name="run_jmeter" description="run jmeter">
    <echo message="Starting to run test: ${jmx.file.fullpath}" /> <!-- ant call param -->
    <copy file="${jmx.file.fullpath}" tofile="${jmx.file.fullpath}.${target.os}" overwrite="true"/>    
    <if>
      <equals arg1="${target.os}" arg2="linux" />
      <then>
        <echo message=" Replacing paths in jmeter data to match the target file system ${target.os}"/>
        <replace file="${jmx.file.fullpath}.${target.os}" token="${windows.local.basepath.location}" value="${linux.local.basepath.location}"/>
        <echo message=" Replacing path delimiters \ into linux delimiter /"/>
        <replace file="${jmx.file.fullpath}.${target.os}" token="\" value="/"/>  
      </then>
    </if>
    <jmeter jmeterhome="${jmeter.install.dir.fullpath}" resultlog="${jmeter.result.file.fullpath}" testplan="${jmx.file.fullpath}.${target.os}" />
    
    <delete file="${jmx.file.fullpath}.${target.os}" />
  </target>
  
  <target name="run_jmeters" description="run all jmeters in dir">
    <echo message="Starting to run test: ${loadtests.basepath}" /> <!-- ant call param -->
    <copy todir="${loadtests.basepath}" overwrite="true">
      <fileset dir="${loadtests.basepath}">
        <include name="${loadtests.extension}"/>
      </fileset>
      <globmapper from="*" to="*.${target.os}"/>
    </copy>
    <if>
      <equals arg1="${target.os}" arg2="linux" />
      <then>
        <echo message=" Replacing paths in jmeter data to match the target file system ${target.os}"/>  
        <replace dir="${loadtests.basepath}" token="${windows.local.basepath.location}" value="${linux.local.basepath.location}">
          <include name="*.${target.os}"/>
        </replace>
        <echo message=" Replacing path delimiters \ into linux delimiter /"/>
        <replace dir="${loadtests.basepath}" value="/">
          <include name="*.${target.os}"/>
          <replacetoken>\</replacetoken>
        </replace>
      </then>
    </if>
    <jmeter jmeterhome="${jmeter.install.dir.fullpath}" resultlog="${jmeter.result.file.fullpath}">
      <testplans dir="${loadtests.basepath}" includes="${loadtests.extension}.${target.os}"/>
    </jmeter>
    
    <delete>
      <fileset dir="${loadtests.basepath}" includes="*.${target.os}"/>
    </delete>
  </target>
  
  
  <target name="make_report" description="make jmeter reports">
    <echo message="Creating test report from: ${jmeter.result.file.fullpath} in: ${jmeter.report.file.fullpath}" />
    <xslt in="${jmeter.result.file.fullpath}" out="${jmeter.report.file.fullpath}" style="${jmeter.install.dir.fullpath}/extras/jmeter-results-detail-report_21.xsl">
      <param name="date" expression="${current.date.time}"/>
      <param name="test_name" expression="CBS3"/>
    </xslt>
  </target>
    
</project>

All files can be downloaded at once using this link.

UPDATE 10/02/2010: new link for full package.