Skip to end of metadata
Go to start of metadata

Background

We want to ensure the robustness and scale ability of all stable Whydah releases

Test set-up

Version tested:

  • Whydah 2.0-rc-2 30.9-2014 15:00-16:00 GST+1

Whydah configuration:

  • Amazon AWS EC2 cloud deployment (region=us-east-1)
    • Amazon Elastic Load Balancer in front of the 10 front-servers running amazon linux - latest version
    • Front-servers: 10 T2 micro servers running SSOLoginWebApp, SecurityTokenService(hazelcast clustered) and TestWebApp with apache proxypass to localhost
    • Backend-server: one of the 10 front servers are also running UAWA and UIB with embedded ApacheDS and HSQLDB (not optimal, as of memory contraints on T2 micro instances)
  • Detailed configuration descriptions

Load-test configuration:

  • Simple loader.io setup
    • Testing two requests in sequence
      • LOGON: /sso/action (Params: redirectURI, name and password)
      • Pick up existing session: /tokenservice/user/xxx/get_usertoken_by_usertokenid (Params: usertokenid (from cookie), apptoken (hardcoded), apptokenid (hardcoded)
    • Manual creation of application session and user session, sessiondata inserted into test
  • Load (1)
    • 1000 clients / test
    • Duration: 1 min
  • Load (2)
    • 250 clients / second
    • Duration: 1 min

Results:

  • average response time: 63 ms / 67 ms
  • average error rate: 0.0 % / 0.0 %
  • Successful responses: 2000 / 20.000
  • Received† 1.98 MB / 20.02 MB
  • Sent 1.41 MB / 13.51 MB

Comments:

  • No undesired behavior observed in logs, server-monitoring or elb-monitoring
  • Manual test simultaneously did work as usual
  • Running the test several times result in similar results
  • Test was done with extensive logging (level: TRACE) less aggressive logging should increase performance

Reflections:

  • By running different loads, we see that a T2 microserver can handle about 200 logon/pick up session request a second. The heavy operation is logon, as it has to go all the way to UIB(singleton). On higher load, the response-time increases (for 500 reqs/s it is 2-3 s).
  • A more realistic test-suite, say 50/1 pickup session/logon should handle 300-400 request/second on T2 micro servers
  • 90% of the Whydah operations scales almost linearly, the exception beeing the few operations which hit the UIB instance.
Labels:
None
Enter labels to add to this page:
Please wait 
Looking for a label? Just start typing.