AppSuite:Load Tests: Difference between revisions

From Open-Xchange
No edit summary
No edit summary
 
(34 intermediate revisions by 2 users not shown)
Line 1: Line 1:
== Software Requirements ==
= OX App Suite Load Tests =
* Java SE JDK6 or JDK7
* Git
* SBT


== Getting Performance Repository ==
We're providing load tests for various OX App Suite APIs and maintain simulations that correspond to specific client implementations. To use it, clone the following repository:
* Cloning Repository
  <code>$ git clone https://git.open-xchange.com/git/wd/testing/performance</code>


* Checkout a branch, e.g. ''develop''
  <code>$ git checkout develop</code>
== Install SBT ==
Since we use the Gatling-SBT plugin, which allows us to use Gatling as a testing framework, we'll need to install SBT first.
=== Installing sbt on Mac ===
* [http://macports.org Macports]
  <code>$ port install sbt</code>
* [http://mxcl.github.com/homebrew Homebrew]
  <code>$ brew install sbt</code>
=== Installing sbt on Windows ===
* Download the [http://dl.bintray.com/sbt/native-packages/sbt/0.13.5/sbt-0.13.5.msi msi installer] and install it.
=== Installing sbt on Linux ===
* [http://dl.bintray.com/sbt/rpm/sbt-0.13.5.rpm RPM] package
* [http://dl.bintray.com/sbt/debian/sbt-0.13.5.deb DEB] package
=== Installing sbt manually ===
* [http://www.scala-sbt.org/0.13/tutorial/Manual-Installation.html Manual] instructions
== Building with SBT ==
After SBT has been successfully installed we can switch to the project's directory:
  <code>$ cd ~/performance/com.openexchange.test.performance.gatling</code>
== Development Environment ==
If you want to execute the tests via an IDE like Eclipse or IntelliJ IDEA then you will also need to install the following software:
* [https://www.eclipse.org Eclipse 4.2] or later
* [http://scala-ide.org Scala IDE plugin]
=== Importing Project into Eclipse ===
# Open Eclipse
# Click on File → Import... → Maven → Existing Maven Project
# Select the gatling folder within the performance repository
== Configurable Parameters ==
List of all available System-Properties and JVM-Arguments:
; protocol: Which protocol to use. ''Possible values are: '''"http"''' or '''"https"'''. Default value: '''"http"'''.''
; host: Which host to use. ''Possible values are: '''"localhost"''', '''"10.0.0.1"''' or if you want to stress several groupware servers, to bypass a loadbalancer for example, then you can use comma-separated IPs or hostnames '''"10.0.0.1,10.0.0.2,localhost"'''. Default value: '''"localhost"'''.''
; users: How many concurrent users to simulate. ''Default value: '''"5"'''.''
; durationTime: Duration time of the simulation. ''Default value: '''"30"'''.''
; durationUnit: The Unit of the durationTime. ''Possible values are: '''"SECONDS"''', '''"MINUTES"''', '''"HOURS"''' or '''"DAYS"'''. Default value: '''"SECONDS"'''.''
; successpercent: Successful responses in percentage. ''Default value: '''"100"'''.''
; reqpersec: Mean number of requests per second (lower bound). ''Default value: '''"0"'''.''
; resptime: Mean response time (lower bound). ''Default value: '''"0"'''.''
; csv_users: Name of the CSV-File for feeding the requests. ''Default value: '''"ox_user_credentials.csv"'''.''
; syncmsgmax: How many objects to synchronize. ''Relevant only for '''QAPerfUSM'''. Default value: '''"50"'''.''
; pimObjects: How many PIM Objects, e.g. contacts, should be created. ''Default value: '''"5"'''.''
=== Snippet from the <properties>-section in the pom.xml ===
  <properties>
    <protocol>http</protocol>
    <host>localhost</host>
    <port>80</port>
    <users>5</users>
    <durationTime>30</durationTime>
    <durationUnit>SECONDS</durationUnit>
    <successpercent>100</successpercent>
    <reqpersec>0</reqpersec>
    <resptime>0</resptime>
    <csv_users>ox_user_credentials.csv</csv_users>
    <syncmsgmax>50</syncmsgmax>
    <pimObjects>5</pimObjects>
  </properties>
=== Snippet from the <jvmArgs>-section in the pom.xml ===
  <jvmArgs>
    <jvmArg>-Dprotocol=${protocol}</jvmArg>
    <jvmArg>-Dhost=${host}</jvmArg>
    <jvmArg>-Dport=${port}</jvmArg>
    <jvmArg>-Dcsv_users=${csv_users}</jvmArg>
    <jvmArg>-Dusers=${users}</jvmArg>
    <jvmArg>-DdurationTime=${durationTime}</jvmArg>
    <jvmArg>-DdurationUnit=${durationUnit}</jvmArg>
    <jvmArg>-Dsuccesspercent=${successpercent}</jvmArg>
    <jvmArg>-Dreqpersec=${reqpersec}</jvmArg>
    <jvmArg>-Dresptime=${resptime}</jvmArg>
    <jvmArg>-Dsyncmsgmax=${syncmsgmax}</jvmArg>
    <jvmArg>-DpimObjects=${pimObjects}</jvmArg>
  </jvmArgs>
== Test Execution ==
=== Pre-Requisites for the Open-Xchange Groupware Server ===
==== Disabling rate limitation ====
* Modify the parameter maxRateLenientClients @server.properties:
  $ vim /opt/etc/open-xchange/etc/server.properties
* Specify a new User-Agent which is excluded from rate limit checks
  "GATLING*"
==== Resources/Data ====
In the data folder you can find an sample csv file which is used to perform a login. If you want to use your own CSV-File place it into the data directory.
* Structure of the csv file:
  <code>
  username,password
  user1,secret
  user2,secret
  user3,secret
  ...
  user998,secret
  user999,secret
  user1000,secret
  </code>
=== Pre-compiled version of the gatling tests ===
Execute the gatling tests via bash script (Linux) or batch file (Windows).
The precompiled version is available here:
https://jenkins.open-xchange.com/view/QA/job/qaPerformanceGatlingTests/lastSuccessfulBuild/artifact/performance/com.openexchange.test.performance.gatling/target/
* Unzip ox-gatling-*-bin.zip
* Change working directory
  $ cd Downloads/ox-gatling-1.0.0-SNAPSHOT
* Please '''adjust''' the ''JAVA_OPTS'' in the scripts ''bin/box-gatling.sh'' or ''bin/ox-gatling.bat'' to match your needs.
** The following example will stress the server with the ''IP 10.0.0.1'' for ''120 seconds'' and hold the load with permanently ''100 user'':
  <code>exec "$JAVACMD" $JAVA_OPTS -Dprotocol=http -Dhost=10.0.0.1 -Dcsv_users=ox_user_credentials.csv -Dusers=100 -DdurationTime=120 -DdurationUnit=SECONDS \
  -classpath "$CLASSPATH" \
  -Dapp.name="ox-gatling" \
  -Dapp.pid="$$" \
  -Dapp.repo="$REPO" \
  -Dapp.home="$BASEDIR" \
  -Dbasedir="$BASEDIR" \
  io.gatling.app.Gatling \
  "$@"</code>
* Execute the script
  $ /bin/bash bin/ox-gatling.sh
<code>
<code>
  Choose a simulation number:
$ git clone https://gitlab.open-xchange.com/qa/performance.git
    [0] com.openexchange.test.performance.gatling.tests.QAPerfOX6
    [1] com.openexchange.test.performance.gatling.tests.QAPerfOXAppSuite
    [2] com.openexchange.test.performance.gatling.tests.QAPerfUSM
    [3] com.openexchange.test.performance.gatling.tests.QAPerfOXDrive
    [4] com.openexchange.test.performance.gatling.tests.QAPerfEAS
</code>
</code>


Now you will see some output of the compilation process but finally you should be able to start the simulation by entering a number.
Documentation can be found at <code>com.openexchange.test.performance.gatling/project/doc</code>. Please checkout the branch that corresponds to the version of OX App Suite to be tested.
 
=== Execute the gatling tests via maven plugin ===
* Switch to the root directory of the gatling project
 
  <code>cd /GIT_REPO/performance/com.openexchange.test.performance.gatling/</code>
 
* Now you can start the performance tests either by using the default values from the pom.xml,
 
  <code>mvn clean gatling:execute</code>
 
or by temporarily passing system properties:
 
  <code>mvn clean gatling:execute -Dhost=localhost -Dcsv_users=ox_user_credentials.csv -Dusers=5 -DdurationTime=20 -DdurationUnit=MINUTES -Dgatling.simulationClass=com.openexchange.test.performance.gatling.tests.QAPerfOX6</code>


== Practical hints and tips ==
[[Category: AppSuite]] [[Category: Testing]]
* If you want to run tests for more than 24h, uncomment the "timeOut" → "simulation" property at gatling.conf and define a value of more than 24h (in seconds).
* If you experience OutOfMemory issues when generating charts, adjust "charting" → "accuracy" to a higher value. This will lower the accuracy of the generated graphs. Especially for long-running tests, this could be set to 100ms.

Latest revision as of 17:19, 11 July 2017

OX App Suite Load Tests

We're providing load tests for various OX App Suite APIs and maintain simulations that correspond to specific client implementations. To use it, clone the following repository:

$ git clone https://gitlab.open-xchange.com/qa/performance.git

Documentation can be found at com.openexchange.test.performance.gatling/project/doc. Please checkout the branch that corresponds to the version of OX App Suite to be tested.