We’re pleased to release beta version 0.5 of Concertio Optimizer which now supports static tuning mode (more information about the changes are available in the Change Log). Earlier this week we’ve shared some results using the new version at the HPC Advisory Council Stanford Conference:
The original slides are available for download from the Conference website. The full video of the presentation is available below:
In the presentation we’ve detailed our experience with the Phoronix Apache Web Server test when running on a Packet type-2 server. The performance in the different phases is detailed in the below graph, where the horizontal axis is the iteration number and the vertical axis is the normalized performance (normalized number of requests per second that were served by the web server):
We ran the benchmark 200 times as follows:
- 20 iterations in baseline – We simply ran the benchmark without Optimizer in the background. The results of this phase were used to normalize the graph.
- 140 iterations in learning phase – We enabled Optimizer in the beginning of this phase with a clean database. The performance in the first 20 iterations (~50 minutes) in this phase were jittery. This is because Optimizer explored various knob options. After around 20 iterations, the performance stabilized at around 23% improvement over the baseline.
- 20 iterations in best phase – We switched Optimizer to “best” mode at the beginning of this phase, which suppresses further exploration and reduces some overhead required for continuous tuning. The performance in this phase averaged 24%.
- 20 iterations in static phase – We switched Optimizer to “static-best” mode at the beginning of this phase. This resulted in Optimizer applying the best settings it had found in the learning phase. After that, Optimizer exited and no longer consumed any CPU cycles or memory. Since dynamic tuning was turned off, the performance improvement dropped from 24% to 8.8%
The full output of the run is available below:
[root@pkt-type2 ~]# concertio-benchmark 20/140/20/20 pts/apache
Set up benchmark suite ...
Start benchmark ...
Baseline Phase : [########################################] 100%
Setting mode to tune
Learning Phase : [########################################] 100%
Setting mode to best
Optimization Phase : [########################################] 100%
Setting mode to static-best
Static Phase : [########################################] 100%
Summary for benchmark pts/apache:
phase, runs, performance, improvement, relative_stdev
Baseline, 20, 27317, 0%, 2.141%
Learning, 140, 32752.9, 19.899%, 5.092%
Optimized, 20, 33940.3, 24.246%, 0.888%
Static, 20, 29730.5, 8.835%, 2.412%
