Background Info & Things to Know¶
The following is a quick start guide to running the Synopsys Optimizer. This guide will go over some of the base software and concepts that are helpful to know for running optimizer-studio. For a live demo, watch the video tutorial
Steps For optimizer-studio Setup:¶
- Download optimizer version for the platform of your choice downloads page
- Login to Conductor from the command line using command
- If you have obtained a license key it can be activated using command
optimizer-license -–activate <license key>
- non license users are able to access the optimizer via community-license, and are limited to two custom knobs per experiment
- Determine desired workload and optimization target
- Create experiment definition file (knobs.yaml), import knobs and/or define custom knobs
- Run optimizer and observe results in Conductor
Configuration Files with YAML Formatting¶
The optimizer uses multiple configuration files to run, and these files use YAML formatting to define the configuration. The files using YAML formatting include the knobs.yaml as well as any imported knobs. Due to having strict syntax, it is recommended to check all yaml format files with a yaml linter after editing. A reliable yaml linter can be found at yamllint.com
Determining the correct workload is the most important part of the optimization process The workload used for the optimization process should be representative of the actual workload we wish to optimize. The desired workload will run for every sample generated by optimizer-studio while attempting to discover the best knob configuration. A typical workload will consist of the bash commands needed to execute the software being optimized, and the commands necessary to determine the optimization metric. It is recommended to know the exact commands needed to execute your workload from the command line. More information about setting up a workload can be found on the workload definition page
Optimizer-studio knobs, also known as the workload tunables, are the parameters that when changed will influence the optimization target of the workload. I.E. increasing the system memory size decreases workload duration – which in this case means the memory size is the tunable and the duration is the optimization target. Optimizer-studio knobs can be defined in several ways, all in a declarative definition language (DSL):
- user-defined in a separate YAML file that can then be imported into experiment definition (knobs.yaml)
- user-defined knobs directly in the experiment definition YAML (knobs.yaml)
- Optimizer-studio has also thousands of pre-defined knobs, a.k.a embedded knobs, that are arranged into libraries, for various software and hardware platforms.
The libraries can be imported with a single
importdirective in experiment definition file.
YAML import example:
This directive will instruct the optimizer to import all Linux OS (system) knobs into the experiment definition. Optimizer-studio will use these definitions to generate configurations and adjustments for each workload sample.
More information about knob formats can be found on the configuration page
Optimization Target Metric¶
The optimization target metric, is the value generated by the workload that optimizer-studio will use as the score to improve the workload. Optimizer-studio provides a pluggable interface for collecting and defining metric data in the experiment definition(knobs.yaml).
The built in plugins are:
- File Plugin - define a file where the workload shall write the metric to
- Shell Plugin - define a collector via shell script within the experiment definition (knobs.yaml)
- APM Tool plugins - NewRelic, AppDynamics, Datadog, etc
For file and shell plugins, metrics are often found in the log outputs generated by the workload. When writing your workload, ensure to select the metric implementation most suitable to you. Further information on retreiving the metric can be on the configuration page Knowing how to extract the desired metric data from the workload is typically the most challenging step in the experiment setup
In the examples provided by the Optimizer, the experiments are run using a run.sh script
Within this script the command
optimizer-studio --stages=readiness,optimization,validation is typically ran
This command starts optimizer-studio as well as selects the stages to run via command line options
It is not necessary to run with a script, the convenience of the script being that it allows for experiment specialization.
Either by setting environment variables before the optimizer-studio run, specifying optimizer command-line options consistently, and other options.
The benefit of the script being the ease of execution.
Once the configuration files are set, it is easy to create the project from the command line. Using the following optimizer-ctl commands will create
a new project with a name and description, as well as automatically adding the new project_guid to the knobs.yaml
optimizer-ctl create_project --name "first_project" --desc "my first example project" --target knobs.yaml
If a project has already been created within the Conductor, the project_guid can be easily provided by hitting the Connect buttone within the project. This value can then be manually copied over to the knobs.yaml. More information about running optimizer-studio can be found on the getting started page