AppSuite:Load Tests
From Open-Xchange
Software Requirements
- Java SE JDK6 or JDK7
- Git
- SBT
Getting Performance Repository
- Cloning Repository
$ git clone https://git.open-xchange.com/git/wd/testing/performance
- Checkout a branch, e.g. develop
$ git checkout develop
Install SBT
Installing sbt on Mac
$ port install sbt
$ brew install sbt
Installing sbt on Windows
- Download the msi installer and install it.
Installing sbt on Linux
Installing sbt manually
- Manual instructions
Development Environment
If you want to execute the tests via an IDE like Eclipse or IntelliJ IDEA then you will also need to install the following software:
- Eclipse 4.2 or later
- Scala IDE plugin
Importing Project into Eclipse
- Open Eclipse
- Click on File → Import... → Maven → Existing Maven Project
- Select the gatling folder within the performance repository
Configurable Parameters
List of all available System-Properties and JVM-Arguments:
- protocol
- Which protocol to use. Possible values are: "http" or "https". Default value: "http".
- host
- Which host to use. Possible values are: "localhost", "10.0.0.1" or if you want to stress several groupware servers, to bypass a loadbalancer for example, then you can use comma-separated IPs or hostnames "10.0.0.1,10.0.0.2,localhost". Default value: "localhost".
- users
- How many concurrent users to simulate. Default value: "5".
- durationTime
- Duration time of the simulation. Default value: "30".
- durationUnit
- The Unit of the durationTime. Possible values are: "SECONDS", "MINUTES", "HOURS" or "DAYS". Default value: "SECONDS".
- successpercent
- Successful responses in percentage. Default value: "100".
- reqpersec
- Mean number of requests per second (lower bound). Default value: "0".
- resptime
- Mean response time (lower bound). Default value: "0".
- csv_users
- Name of the CSV-File for feeding the requests. Default value: "ox_user_credentials.csv".
- syncmsgmax
- How many objects to synchronize. Relevant only for QAPerfUSM. Default value: "50".
- pimObjects
- How many PIM Objects, e.g. contacts, should be created. Default value: "5".
Snippet from the <properties>-section in the pom.xml
<properties> <protocol>http</protocol> <host>localhost</host> <port>80</port> <users>5</users> <durationTime>30</durationTime> <durationUnit>SECONDS</durationUnit> <successpercent>100</successpercent> <reqpersec>0</reqpersec> <resptime>0</resptime> <csv_users>ox_user_credentials.csv</csv_users> <syncmsgmax>50</syncmsgmax> <pimObjects>5</pimObjects> </properties>
Snippet from the <jvmArgs>-section in the pom.xml
<jvmArgs>
<jvmArg>-Dprotocol=${protocol}</jvmArg>
<jvmArg>-Dhost=${host}</jvmArg>
<jvmArg>-Dport=${port}</jvmArg>
<jvmArg>-Dcsv_users=${csv_users}</jvmArg>
<jvmArg>-Dusers=${users}</jvmArg>
<jvmArg>-DdurationTime=${durationTime}</jvmArg>
<jvmArg>-DdurationUnit=${durationUnit}</jvmArg>
<jvmArg>-Dsuccesspercent=${successpercent}</jvmArg>
<jvmArg>-Dreqpersec=${reqpersec}</jvmArg>
<jvmArg>-Dresptime=${resptime}</jvmArg>
<jvmArg>-Dsyncmsgmax=${syncmsgmax}</jvmArg>
<jvmArg>-DpimObjects=${pimObjects}</jvmArg>
</jvmArgs>
Test Execution
Pre-Requisites for the Open-Xchange Groupware Server
Disabling rate limitation
- Modify the parameter maxRateLenientClients @server.properties:
$ vim /opt/etc/open-xchange/etc/server.properties
- Specify a new User-Agent which is excluded from rate limit checks
"GATLING*"
Resources/Data
In the data folder you can find an sample csv file which is used to perform a login. If you want to use your own CSV-File place it into the data directory.
- Structure of the csv file:
username,password
user1,secret
user2,secret
user3,secret
...
user998,secret
user999,secret
user1000,secret
Pre-compiled version of the gatling tests
Execute the gatling tests via bash script (Linux) or batch file (Windows).
The precompiled version is available here: https://jenkins.open-xchange.com/view/QA/job/qaPerformanceGatlingTests/lastSuccessfulBuild/artifact/performance/com.openexchange.test.performance.gatling/target/
- Unzip ox-gatling-*-bin.zip
- Change working directory
$ cd Downloads/ox-gatling-1.0.0-SNAPSHOT
- Please adjust the JAVA_OPTS in the scripts bin/box-gatling.sh or bin/ox-gatling.bat to match your needs.
- The following example will stress the server with the IP 10.0.0.1 for 120 seconds and hold the load with permanently 100 user:
exec "$JAVACMD" $JAVA_OPTS -Dprotocol=http -Dhost=10.0.0.1 -Dcsv_users=ox_user_credentials.csv -Dusers=100 -DdurationTime=120 -DdurationUnit=SECONDS \
-classpath "$CLASSPATH" \
-Dapp.name="ox-gatling" \
-Dapp.pid="$$" \
-Dapp.repo="$REPO" \
-Dapp.home="$BASEDIR" \
-Dbasedir="$BASEDIR" \
io.gatling.app.Gatling \
"$@"
- Execute the script
$ /bin/bash bin/ox-gatling.sh
Choose a simulation number:
[0] com.openexchange.test.performance.gatling.tests.QAPerfOX6
[1] com.openexchange.test.performance.gatling.tests.QAPerfOXAppSuite
[2] com.openexchange.test.performance.gatling.tests.QAPerfUSM
[3] com.openexchange.test.performance.gatling.tests.QAPerfOXDrive
[4] com.openexchange.test.performance.gatling.tests.QAPerfEAS
Now you will see some output of the compilation process but finally you should be able to start the simulation by entering a number.
Execute the gatling tests via maven plugin
- Switch to the root directory of the gatling project
cd /GIT_REPO/performance/com.openexchange.test.performance.gatling/
- Now you can start the performance tests either by using the default values from the pom.xml,
mvn clean gatling:execute
or by temporarily passing system properties:
mvn clean gatling:execute -Dhost=localhost -Dcsv_users=ox_user_credentials.csv -Dusers=5 -DdurationTime=20 -DdurationUnit=MINUTES -Dgatling.simulationClass=com.openexchange.test.performance.gatling.tests.QAPerfOX6
Practical hints and tips
- If you want to run tests for more than 24h, uncomment the "timeOut" → "simulation" property at gatling.conf and define a value of more than 24h (in seconds).
- If you experience OutOfMemory issues when generating charts, adjust "charting" → "accuracy" to a higher value. This will lower the accuracy of the generated graphs. Especially for long-running tests, this could be set to 100ms.