Difference between revisions of "AppSuite:Load Tests"

(Build and execute the tests with the SBT-Plugin)
(Snippet from the -section in the pom.xml)
Line 78: Line 78:
  
 
; pimObjects: How many PIM Objects, e.g. contacts, should be created. ''Default value: '''"5"'''.''
 
; pimObjects: How many PIM Objects, e.g. contacts, should be created. ''Default value: '''"5"'''.''
 
=== Snippet from the <properties>-section in the pom.xml ===
 
  <properties>
 
    <protocol>http</protocol>
 
    <host>localhost</host>
 
    <port>80</port>
 
    <users>5</users>
 
    <durationTime>30</durationTime>
 
    <durationUnit>SECONDS</durationUnit>
 
    <successpercent>100</successpercent>
 
    <reqpersec>0</reqpersec>
 
    <resptime>0</resptime>
 
    <csv_users>ox_user_credentials.csv</csv_users>
 
    <syncmsgmax>50</syncmsgmax>
 
    <pimObjects>5</pimObjects>
 
  </properties>
 
  
 
=== Snippet from the <jvmArgs>-section in the pom.xml ===
 
=== Snippet from the <jvmArgs>-section in the pom.xml ===

Revision as of 14:18, 29 July 2014

Software Requirements

  • Java SE JDK6 or JDK7
  • Git
  • SBT

Getting Performance Repository

  • Cloning Repository
 $ git clone https://git.open-xchange.com/git/wd/testing/performance
  • Checkout a branch, e.g. develop
 $ git checkout develop

Install SBT

Since we use the Gatling-SBT plugin, which allows us to use Gatling as a testing framework, we'll need to install SBT first.

Installing sbt on Mac

 $ port install sbt
 $ brew install sbt

Installing sbt on Windows

Installing sbt on Linux

Installing sbt manually

Build and execute the tests with the SBT-Plugin

After SBT has been successfully installed we can switch to the project's directory:

 $ cd ~/performance/com.openexchange.test.performance.gatling


SBT can be used both as a command line script and as a build console. Most of the commands can be run standalone by passing the command as an argument to SBT, e.g.

 $ sbt update

Note that if a command takes arguments, you need to quote the entire argument path, e.g.

 $ sbt 'gatling:testOnly com.openexchange.test.performance.gatling.tests.QAPerfOX6'

Development Environment

If you want to execute the tests via an IDE like Eclipse or IntelliJ IDEA then you will also need to install the following software:

Importing Project into Eclipse

  1. Open Eclipse
  2. Click on File → Import... → Maven → Existing Maven Project
  3. Select the gatling folder within the performance repository

Configurable Parameters

List of all available System-Properties and JVM-Arguments:

protocol
Which protocol to use. Possible values are: "http" or "https". Default value: "http".
host
Which host to use. Possible values are: "localhost", "10.0.0.1" or if you want to stress several groupware servers, to bypass a loadbalancer for example, then you can use comma-separated IPs or hostnames "10.0.0.1,10.0.0.2,localhost". Default value: "localhost".
users
How many concurrent users to simulate. Default value: "5".
durationTime
Duration time of the simulation. Default value: "30".
durationUnit
The Unit of the durationTime. Possible values are: "SECONDS", "MINUTES", "HOURS" or "DAYS". Default value: "SECONDS".
successpercent
Successful responses in percentage. Default value: "100".
reqpersec
Mean number of requests per second (lower bound). Default value: "0".
resptime
Mean response time (lower bound). Default value: "0".
csv_users
Name of the CSV-File for feeding the requests. Default value: "ox_user_credentials.csv".
syncmsgmax
How many objects to synchronize. Relevant only for QAPerfUSM. Default value: "50".
pimObjects
How many PIM Objects, e.g. contacts, should be created. Default value: "5".

Snippet from the <jvmArgs>-section in the pom.xml

 <jvmArgs>
   <jvmArg>-Dprotocol=${protocol}</jvmArg>
   <jvmArg>-Dhost=${host}</jvmArg>
   <jvmArg>-Dport=${port}</jvmArg>
   <jvmArg>-Dcsv_users=${csv_users}</jvmArg>
   <jvmArg>-Dusers=${users}</jvmArg>
   <jvmArg>-DdurationTime=${durationTime}</jvmArg>
   <jvmArg>-DdurationUnit=${durationUnit}</jvmArg>
   <jvmArg>-Dsuccesspercent=${successpercent}</jvmArg>
   <jvmArg>-Dreqpersec=${reqpersec}</jvmArg>
   <jvmArg>-Dresptime=${resptime}</jvmArg>
   <jvmArg>-Dsyncmsgmax=${syncmsgmax}</jvmArg>
   <jvmArg>-DpimObjects=${pimObjects}</jvmArg>
 </jvmArgs>

Test Execution

Pre-Requisites for the Open-Xchange Groupware Server

Disabling rate limitation

  • Modify the parameter maxRateLenientClients @server.properties:
 $ vim /opt/etc/open-xchange/etc/server.properties
  • Specify a new User-Agent which is excluded from rate limit checks
 "GATLING*"

Resources/Data

In the data folder you can find an sample csv file which is used to perform a login. If you want to use your own CSV-File place it into the data directory.

  • Structure of the csv file:
 
 username,password
 user1,secret
 user2,secret
 user3,secret
 ...
 user998,secret
 user999,secret
 user1000,secret
 

Practical hints and tips

  • If you want to run tests for more than 24h, uncomment the "timeOut" → "simulation" property at gatling.conf and define a value of more than 24h (in seconds).
  • If you experience OutOfMemory issues when generating charts, adjust "charting" → "accuracy" to a higher value. This will lower the accuracy of the generated graphs. Especially for long-running tests, this could be set to 100ms.