Overview¶
The below describes the required steps to get started with using Concertio Optimizer Studio software.
While Optimizer Studio can run under any modern Linux distribution, the directions below have been tested on Ubuntu 20.04, Ubuntu 18.04, Ubuntu 16.04, Centos 7 and CentOS 8.
Community Edition¶
The community edition does not require activating a license.
Instead, it requires connecting to Concertio's Experiment Management System (a software as a service).
Following are the steps to set up optimizer-studio to connect to that service:
- sign up at optimizer.concertio.com
- login to the service and create a new Project
- once project created, click the Connect button and follow the popup instructions
For instructions how to run experiments, please skip the license activation section, straight to Using Optimizer Studio.
Activating Your License (paid plans only)¶
Prior to using Optimizer Studio, you'll have to activate your license. In order to do so you'll need to use the optimizer-license tool:
$ optimizer-license
Concertio Optimizer License Handling Tool
Online documentation is available at https://www.concertio.com/docs/
Usage: optimizer-license ACTION [--verbose]
Supported ACTIONs:
--help Display this help message
--provider LICENSE PROVIDER Choose the provider of your license key [cryptlex (default) | synopsys]
--activate PRODUCT KEY Online Activation of your installation using the product key [PRODUCT KEY]
--deactivate Online Deactivation of your installation
--activation-request PRODUCT KEY Generate an offline activation request file named <activation.req> using the product key [PRODUCT KEY]
--activate-offline PATH Offline activation using the activation response file [PATH]
--deactivate-offline Generate an offline deactivation request file <deactivation.req>
--show Display license information
Optional:
--verbose Detailed output.
If your system is connected to the internet you can use the online activation:
$ optimizer-license --activate XXXXXX-XXXXXX-XXXXXX-XXXXXX-XXXXXX-XXXXXX
(XXXXXX-XXXXXX-XXXXXX-XXXXXX-XXXXXX-XXXXXX is the license key you received when you purchased Optimizer Studio).
Otherwise, if your system is offline the procedure is as follows:
First you'll need to generate a request file:
$ optimizer-license --activation-request XXXXXX-XXXXXX-XXXXXX-XXXXXX-XXXXXX-XXXXXX
This will generate a request file named activation.req, you should copy this file and send it as attachment by email to Synopsys ([email protected]).
Next, in a reply email you'll receive an activation file. You should copy the file to your system and execute the command:
$ optimizer-license --activate-offline FILE_PATH
(FILE_PATH is the file path of your activation file).
NOTE: you can use --provider cryptlex action, but cryptlex license provider set as a default, so it is not mandatory.
If you use Synopsys licensing you can activate the product in two ways:
- Using online activation:
$ optimizer-license --provider synopsys --activate <PORT@SERVER>
(PORT number and SERVER ip is the license server location you received when you purchased Optimizer Studio from Synopsys).
- Using license file:
$ optimizer-license --provider synopsys --activate FILE_PATH
(FILE_PATH is the file path of the activation file).
The SCL data (<PORT>@<SERVER> or /path/to/file.lic) will be saved in ~/.concertio/credentials.yaml file.
Deactivation¶
In case you wish to deactivate your license (for example, if you wish to use your license on a different system), you should use the following command (if your system is online):
$ optimizer-license --deactivate
NOTE: For Synopsys licensing, this command will deactivate both types of licenses, online licenses and licenses that were activated by file.
Alternatively, if your system is offline, you should use:
$ optimizer-license deactivate-offline
This will generate a deactivation request file named deactivation.req. You should email this file to Concertio.
Using Optimizer Studio¶
For a brief description of available command line parameters, issue:
$ optimizer-ctl --help
NAME:
Concertio Optimizer Studio - version 3.14.0 (build time: 2021-12-27_09:56_[UTC])
USAGE:
optimizer-ctl [OPTION] [OPTION] ... "<workload> [workload args]"
optimizer-ctl [OPTION] [OPTION] ...
VERSION:
3.25.0
DESCRIPTION:
Online documentation is available at https://www.concertio.com/docs/
AUTHOR:
Concertio
COMMANDS:
help, h Shows a list of commands or help for one command
GLOBAL OPTIONS:
create_project Create a new experiment project
download_project Download project files from Conductor
import_accel_knobs Extracts acceleration knobs from optimizer-studio report
login Log in to Conductor
logout Log out from Conductor
nats_server Control over NATS service
status Print the status of current running task
studio_log Configure Studio log format
set_baseline Download a specific configuration from Conductor
apply_config Get a specific configuration from Conductor, and select an action
analyze Traverse over all the options of a knob, or all the knobs
sample sample given configuration N times
refine refine (i.e. exclude low or no impact knobs) a given knob configuration
revoke Revoke a pending action
init Initialize experiment
resume Resume optimization from previous Optimizer Studio run
start Execute experiment
stop Stop execution of experiment
run Initialize and execute experiment
testknob Test an optimization knob
list, ls List existing experiments
remove, rm Remove experiment by ID from Conductor and locally
help, h Shows a list of commands or help for one command
SUB_OPTIONS:
Each command has sub-options which can be listed in the following format:
optimizer-ctl [GLOBAL_OPTION] --help
COPYRIGHT:
Copyright (c) 2016-2021, Concertio
Running Optimizer Studio¶
In order to optimize a workload, the simplest form is to provide a workload script as a parameter, as follows:
$ optimizer-ctl init --stages=optimization,done
$ optimizer-ctl start
Concertio Optimizer Studio version 3.5.0
Optimizing ./my_workload
Optimization target: duration
Starting runtime (no time limit) ...
...
Optimization time: 00:06:23
Progress: 100%
Best known knob settings so far:
vm.swappiness: 60
sys.vm.overcommit_ratio: 40
Settings written to: optimizer_studio_settings.sh
In this example, the script my_workload is optimized by Optimizer Studio. Since no configuration file is specified using --knobs <file>.yaml, Optimizer Studio defaults to optimizing CPU and OS knobs, unless you already have a predefined knobs.yaml in current folder. Optimizing CPU and OS knobs requires root access, so this is why Optimizer Studio is executed using sudo in the above example.
However, the more recommended way to optimize workloads, is to use Optimizer-Studio declarative workload definition in the experiment definition file (knobs.yaml). More information can be found in the workload defintition page.
There are three main ways to use Optimizer Studio to optimize a workload:
- Synchronous Sampling (Optimizing Full Workloads) - This is the default optimization mode. Optimizer Studio will run the workload until completion several times and will attempt to minimize its runtime or maximize its reported metric using different system settings. This method is appropriate for complex workloads.
- Asynchronous Sampling (Optimizing Specific Metrics) - By specifying
kind: asyncin the workload defintition section of theknobs.yaml, Optimizer Studio will explore the best settings for an on-going workload. This method is appropriate for background workloads, where early feedback of application performance metrics can be used to run many more samples, asynchornously to the workload. - In Situ Sampling (Optimizing Application Services) - By specifying
kind: accelin the workload defintition section of theknobs.yaml, Optimizer Studio will explore the best settings for online services, in an event-driven fashion. This method is appropriate for daeamons, micro-service and services not started by Optimizer-Studio, by injecting Concertio Accelerator to the services startup command. More information about In-Situ Optimization can be found hereunder at the "In-situ Optimization" section.
The specifics of the optimization method can be controlled using the configuration files.
Workload Settings¶
Optimizer Studio is responsible for running the workload provided by workload script (workload:->start_command section in knobs.yaml or via command line). There are several command line parameters that control the way Optimizer Studio runs the workload:
--max-minutes. The maximum optimization time can be limited by this parameter. If it's not specified or is zero, Optimizer Studio will terminate the optimization process according to its estimations.
--settings-script. After optimization, Optimizer Studio outputs the discovered optimal settings and the baseline settings to this file, which can run as a script.
Running workloads as a non-root user while optimizing OS/CPU knobs¶
Optimizer Studio requires root access when optimizing OS and CPU knobs. It is possible to run the workload as the current user using su as in the following example:
In the workload definition style:
workload:
run:
command: su - $USER -c "./workload.sh"
Configuring Knobs¶
Optimizer Studio can optimize user-defined knobs by providing a configuration file. This is explained in the Configuration section.
Running Long Workloads¶
Status Report¶
During optimization session, Optimizer Studio prints out a short line about its progress. If a user wants to get more detailed information about optimization session status, they can open another terminal session and run the following command:
$ optimizer-ctl status
Workload Timeout¶
Sometimes, a workload script can run too long time or even indefinitely. In such cases a user might want to limit a single workload run time with the workload-timeout option in the knobs.yaml. In case that workload script runs more than the allowed time, Optimizer studio will kill it using SIGKILL signal, and the corresponding knob settings will be invalidated.
The format for time specification is AhBmCs where A, B, C specify correspondingly hours, minutes and seconds. For example:
workload:
kind: sync
run:
command: ./workload.sh
stop: pkill "..."
timeout: 10m
Further information on workload timeout can be found here.
Resuming Stopped Optimization Session¶
Sometimes, when optimization session stops in the middle, a user may want to resume it from the
same point it's stopped. optimizer-ctl resume is one way to achieve this. Another way is
to "replay" the data collected during the previous optimization session.
Optimizer Studio stores the optimization data in ${HOME}/.concertio/{EXPERIMENTID}/opt.data. By default, upon invocation of Optimizer Studio, this file is removed. In order to retain the data from the previous run, the resume command line parameter can be used with the EXPERIMENT-ID flag:
$ optimizer-ctl resume --id [EXPERIMENT-ID]
The EXPERIMENT-ID value can be found in Conductor with the shorthand version being found in the original experiment name. You can provide a full EXPERIMENT-ID or only an initial part of it - Optimizer Studio will identify the experiment correctly, as long as it is unambiguous. In case you omit the EXPERIMENT-ID altogether, the last (most recently stopped) experiment will be resumed.
User can also add stages to the resume command to control which specific stages are of interest from which to continue the experiment. For example:
$ optimizer-ctl resume
$ optimizer-ctl init --stages=refinement,validation
Notice: In case you omit the EXPERIMENT-ID altogether, the last (most recently stopped) experiment will be assumed.
Replaying Optimization Samples¶
During the optimization, Optimizer Studio stores all metrics collected for each knob configuration applied in a CSV file. This file is stored in ${HOME}/.concertio/{experimentID} directory and has the name metrics_<timestamp>.csv, where timestamp specifies the file creation time at the beginning of optimization session. Using the replay command line parameter, a user can feed all this data to optimizer:
$ optimizer-ctl init --knobs <my_knobs.yaml> --replay ${HOME}/.concertio/{experimentID}/<my_metrics>.csv
When To Use Retain And Replay¶
Each method of resuming optimization has its pros and cons.
Retaining Optimization Data allows to restore the optimization session exactly in the same state as it was when the optimization has stopped. However, if there were any changes in the environment, knobs definitions or product version, Optimizer Studio may fail to read this information, and then it discards it altogether. In this case the optimization starts from the very beginning.
Replaying metrics.csv uses the data in this file. Prior to invoking Optimizer Studio, users can review it, modify, drop some invalid samples, etc. Replaying the CSV file is a fast operation. However the optimizer state after replay is not a precise copy of its state prior to optimization stop. Following this, optimizer may decide to reevaluate some knob configurations, which may take additional time to run the workload. However, this overhead is usually relatively small.
In-situ Optimization¶
Note: This is an evolving new feature. Expect changes and additions to this feature.
In order to achieve best results, it is preferred to perform the optimization on a production machine rather than in a simulated laboratory environment. This is of course not always possible, but when it is (e.g., when the optimization may take place on a single production server) it may yield better results.
In-situ optimization opens up the possibility for new optimization techniques such as replacing memory allocation algorithms in the optimized services.
Configuring In-situ Optimization¶
Assuming your service start-up looks like the following:
MyService <Param1> <Param2> ... <ParamN>
Adding the wrapper command accelerate will mark the service as a target for optimization:
accelerate MyService <Param1> <Param2> ... <ParamN>
Next you have to configure a suitable knobs.yaml file. A simple knobs.yaml file may look like:
global_settings:
min_samples_per_config: 1
min_baseline_samples: 1
domain:
common:
knobs:
memory:
kind: accel.mem_alloc
options: [\off\, mimalloc, tcmalloc, jemalloc]
This simple configuration file will select the best memory allocation algorithm for MyService among the default glibc, mimalloc, tcmalloc and jemalloc algorithms. The \off\ option is unique to accel.* kind of knobs. It is a special option which means do-nothing.
In order to start the optimization session, optimizer-studio needs to be started with accelerate workload. accelerate is a reserved workload name which is used for in-situ optimization:
optimizer-studio accelerate
When optimizer-studio is running in accelerate mode, it will wait for an instance of MyService to show up in order to start each sample. This means that your timing of the service operation will not be affected by the optimization process (i.e., if the service is started once every 24 hours, then each sample will take 24 hours). If your service is not regularly restarted, it is advised to add a cron job (or some kind of script) which will force a restart of the service. This will allow optimizer-studio to evaluate different configurations.
Applying In-situ Knobs to specific applications¶
While the default behavior is to apply each in-situ knob to any accelerated application, it is possible to restrict the application of each knob to a specific application (or a group of applications).
This can be done by adding the filter field to the knob definition. The value of this field is a regex which needs to be satisfied by the accelerated app name in order for the knob to be applicable to the accelerated app. For example:
global_settings:
min_samples_per_config: 1
min_baseline_samples: 1
domain:
common:
knobs:
memory:
kind: accel.mem_alloc
options: [\off\, mimalloc, tcmalloc, jemalloc]
filter: MyApp
This knob definition will be applicable to any app name which satisfies "MyApp" - for example MyApp, MyApp2 and ThisIsMyApp.
Available In-situ Knobs¶
The following kinds of knobs are available for use during In-situ optimization:
1. Memory allocators¶
...
knobs:
my_memory_knob:
kind: accel.mem_alloc
options: [\off\, mimalloc, tcmalloc, jemalloc]
2. Environment variables injection¶
...
knobs:
SOME_VAR:
kind: accel.env
options: [\off\, some_value, some_other_value]
Note: The knob name (SOME_VAR in the above example) is the name of the environment variable you wish to set.
3. Parameter injection¶
...
knobs:
my_param_knob:
kind: accel.param
options: [\off\, "-some_param some_value"]
Note: Currently parameter placement options are not supported. The parameters will be added at the begining of the application parameter list.
4. Acceleration libraries¶
Currently there are 2 acceleration libraries: pid and log acceleration and their configuration may look like:
...
knobs:
my_pid_knob:
kind: accel.lib
options: [\off\, pid]
my_log_knob:
kind: accel.lib
options: [\off\, "log path=/my/log/path.log bufsize=65536", "log path=/my/log/path.log bufsize=131072"]
Note: The log library takes 2 parameters: path and bufsize. path is the full file path of the accelerated application log and bufsize is the size of memory buffer to use during optimization stated in bytes.
Using Optimization Results¶
Optimizer studio stores the results of the optimization in a script, as configured in settings.yaml. This script can be used to apply the discovered results:
$ /tmp/studio-settings.XYZ.sh tuned
Concertio Optimizer Studio, version 3.5.0
Auxiliary script for loading optimized settings for ./my_workload
Creation date: Thu May 18 13:35:58 UTC 2021
Apply tuned settings
...
To revert to baseline, use:
$ /tmp/studio-settings.XYZ.sh baseline
Concertio Optimizer Studio, version 3.5.0
Auxiliary script for loading optimized settings for ./my_workload
Creation date: Thu May 18 13:35:58 UTC 2021
Applying baseline settings
...
To define bash variable array with the knob values, use:
$ . /tmp/studio-settings.XYZ.sh valarray
Concertio Optimizer Studio, version 3.5.0
Auxiliary script for loading optimized settings for ./my_workload
Creation date: Thu May 18 13:35:58 UTC 2021
Exporting values as KNOB_VALUES_TUNED and KNOB_VALUES_BASELINE array variables
$ echo ${KNOB_VALUES_TUNED[@]}
...