summaryrefslogtreecommitdiff
path: root/README.rst
blob: 7aaaa14cefdc89293a34a26733cdd2a8cb0e69cf (plain)
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
soledad performance tests
-------------------------
some minimalistic benchmarks.
small prototypes to compare async behavior under load.

intended to evolve to small prototypes that:

* run soledad
* try different options for the decryption pool (inline, threads, processes).
* serve also a simple server to test how reactor is able to respond under cpu
  load.


You can use the makefile to launch different servers and compare their
performance::
  make inline-server  # executes cpu load inline
  make thread-server  # cpu load in a twisted threadpool
  make ampoule-server # cpu load in an ampoule process pool
  make perf           # runs httperf against the server, with a moderate cpu load.
  make perf-little    # runs httperf against the server, (less requests).
  make perf-easy      # runs httperf against a no-cpu load.

If you want to modify the cpu load, you can pass the FIB parameter as an
environment variable::
  FIB=20 make inline-server
  curl localhost:8080/
  $ answer is >>> 6765


Analysis
---------------
Let's run some experiments, in the three situations.

A) Compare a **fixed, moderate cpu load** (ie, parameters to the fib function in the range fib(25)/fib(30)) in terms of req/sec.

* Very similar rates. For fib(30), this gives something ~3 req/s in my machine.
* some overhead is appreciated.
* RAM usage??
* graph it!!

B) Stablish a **baseline for the no-cpu** perf case (perf-easy). In my machine this varies, but
   it's in the range 600-800 req/s. Note: since w/o cpu load this target runs very
   fas, this can be scripted to log one measure ever 500ms or so.

c) **Simultaneous easy+load**: Observe how the no-cpu perf case degrades when run
   against each one of the three servers, *while the servers are handling a moderate cpu load*.
   Still have to graph this properly, and measure std etc, but looking quickly
   at data I have three conclusions (yes, I'm biased!).

   * inline cpu load is a no-go: it blocks the reactor.
   * threaded is better,
   * but ampoule-multiprocessing is superior when we look at how responsive the reactor is still.

   to do still for this case:

   * RAM usage?? graph before, during, after
   * graph it!!
   * experiment with different parameters for the process pool.


Run debug soledad server
-------------------------
need patched branch::
  twistd -n web --port 2323 --wsgi leap.soledad.server.debug_local_application_do_not_use 


To-Do
--------------
* [x] make the cpu load variable (parameter to fib function: pass it as env var).
* [x] graph req/sec in response to variable cpu loads (parameter to fib).
* [ ] graph response of perf-easy DURING a run of perf/perf-little. 
* [ ] compare the rate of responsiveness against variable cpu loads.
* [ ] scale these minimalistic examples to realistic payload decryption using gnupg.