Runner¶
The Runner is used to automatically do closed-loop optimization. NeptuneRunner can connect to the Neptune experiment tracker
-
class
summit.run.
Runner
(strategy: summit.strategies.base.Strategy, experiment: summit.experiment.Experiment, num_initial_experiments=None, max_iterations=100, batch_size=1, f_tol=1e-05, max_same=None, max_restarts=0, **kwargs)[source]¶ Run a closed-loop strategy and experiment cycle
- Parameters
strategy (
Strategy
) – The summit strategy to be used. Note this should be an object (i.e., you need to call the strategy and then pass it). This allows you to add any transforms, options in advance.experiment (
Experiment
) – The experiment or benchmark class to use for running experimentsmax_iterations (int, optional) – The maximum number of iterations to run. By default this is None.
num_initial_experiments (int, optional) – Number of initial experiments to run, if different than batch size. Default is to start with batch size.
batch_size (int, optional) – The number experiments to request at each call of strategy.suggest_experiments.
f_tol (float, optional) – How much difference between successive best objective values will be tolerated before stopping. This is generally useful for nonglobal algorithms like Nelder-Mead. Default is None.
max_same (int, optional) – The number of allowed iterations where the objectives don’t improve by more than f_tol. Default is None.
max_restarts (int, optional) – Number of restarts if max_same where is violated. Default is 0.
Examples
>>> from summit import * >>> benchmark = SnarBenchmark() >>> strategy = Random(benchmark.domain) >>> r = Runner(strategy=strategy, experiment=benchmark, max_iterations=10) >>> # Turn progress bar on by setting to True below >>> r.run(progress_bar=False)
-
run
(**kwargs)[source]¶ Run the closed loop experiment cycle
- Parameters
prev_res (DataSet, optional) – Previous results to initialize the optimization
save_freq (int, optional) – The frequency with which to checkpoint the state of the optimization. Defaults to None.
save_at_end (bool, optional) – Save the state of the optimization at the end of a run, even if it is stopped early. Default is True.
save_dir (str, optional) – The directory to save checkpoints locally. Defaults to not saving locally.
-
class
summit.run.
NeptuneRunner
(strategy: summit.strategies.base.Strategy, experiment: summit.experiment.Experiment, neptune_project: str, neptune_experiment_name: str, neptune_tags: Optional[list] = None, neptune_description: Optional[str] = None, neptune_files: Optional[list] = None, hypervolume_ref=None, **kwargs)[source]¶ Run a closed-loop strategy and experiment cycle with logging to Neptune
- Parameters
strategy (
Strategy
) – The summit strategy to be used. Note this should be an object (i.e., you need to call the strategy and then pass it). This allows you to add any transforms, options in advance.experiment (
Experiment
) – The experiment or benchmark class to use for running experimentsneptune_project (str) – The name of the Neptune project to log data to
neptune_experiment_name (str) – A name for the neptune experiment
netpune_description (str, optional) – A description of the neptune experiment
files (list, optional) – A list of filenames to save to Neptune
max_iterations (int, optional) – The maximum number of iterations to run. By default this is 100.
batch_size (int, optional) – The number experiments to request at each call of strategy.suggest_experiments. Default is 1.
f_tol (float, optional) – How much difference between successive best objective values will be tolerated before stopping. This is generally useful for nonglobal algorithms like Nelder-Mead. Default is None.
max_same (int, optional) – The number of iterations where the objectives don’t improve by more than f_tol. Default is max_iterations.
max_restarts (int, optional) – Number of restarts if f_tol is violated. Default is 0.
hypervolume_ref (array-like, optional) – The reference for the hypervolume calculation if it is a multiobjective problem. Should be an array of length the number of objectives. Default is at the origin.
-
run
(**kwargs)[source]¶ Run the closed loop experiment cycle
- Parameters
save_freq (int, optional) – The frequency with which to checkpoint the state of the optimization. Defaults to None.
save_at_end (bool, optional) – Save the state of the optimization at the end of a run, even if it is stopped early. Default is True.
save_dir (str, optional) – The directory to save checkpoints locally. Defaults to ~/.summit/runner.