finantic.Optimizer is an optimizer extension package for WealthLab 7. The package contains the following optimizer algorithms.

Optimization becomes a challenge if there are many parameters:

Random search optimizer initializes random parameters between min and max of the provided parameters.

Roughly based on: http://www.jmlr.org/papers/volume13/bergstra12a/bergstra12a.pdf

## Random Search

#### Parameters

**iterations:** The number of iterations to perform

**seed**: Seed for random number generator

## Grid Search

Simple grid search that tries all combinations of the provided parameters.

if 1 parameter: create a vector with i*terations* values

if 2 parameters: create an array with values, i.e. each axis has sqrt(*iterations*) values

if 3 parameters: each axis has *iterations*^(1/3) values

etc…

#### Parameters

**iterations:** The number of iterations to perform

## Particle Swarm

Particle Swarm optimizer (PSO). PSO is initialized with a group of random particles

and then searches for optima by updating generations. In every iteration, each particle is updated by following two „best“ values. The first one is the best solution found by the specific particle so far. The other „best“ value is the global best value obtained by any particle in the population so far.

http://www.swarmintelligence.org/tutorials.php

https://en.wikipedia.org/wiki/Particle_swarm_optimization

## Globalized Bounded Nelder-Mead

Globalized bounded Nelder-Mead method. This version of Nelder-Mead optimization

avoids some of the shortcomings the standard implementation.

Specifically it is better suited for multi-modal optimization problems through its restart property.

It also respect the bounds given by the provided parameter space.

Roughly based on:

http://home.ku.edu.tr/~daksen/2004-Nelder-Mead-Method-Wolff.pdf

and

http://www.emse.fr/~leriche/GBNM_SMO_1026_final.pdf

#### Parameters

**maxRestarts**: Maximum number of restart (default is 8)**noImprovementThreshold:** Minimum value of improvement before the improvement is accepted as an actual improvement (default is 0.001)**maxIterationsWithoutImprovement:** Maximum number of iterations without an improvement (default is 5)**maxIterationsPrRestart:** Maximum iterations pr. restart. 0 is no limit and will run to convergence (default is 0)**maxFunctionEvaluations:** Maximum function evaluations. 0 is no limit and will run to convergence (default is 0)**alpha:** Coefficient for reflection part of the algorithm (default is 1)**gamma:** Coefficient for expansion part of the algorithm (default is 2)**rho:** Coefficient for contraction part of the algorithm (default is -0.5)**sigma:** Coefficient for shrink part of the algorithm (default is 0.5)**seed:** Seed for random restarts**maxDegreeOfParallelism:** Maximum number of concurrent operations (default is -1 (unlimited))

## Bayesian Optimization

Bayesian optimization (BO) for global black box optimization problems. BO learns a model based on the initial parameter sets and scores.

This model is used to sample new promising parameter candidates which are evaluated and added to the existing parameter sets.

This process iterates several times. The method is computational expensive so is most relevant for expensive problems,

where each evaluation of the function to minimize takes a long time, like hyper parameter tuning a machine learning method.

But in that case it can usually reduce the number of iterations required to reach a good solution compared to less sophisticated methods.

Implementation loosely based on:

http://www.cs.ubc.ca/~hutter/papers/10-TR-SMAC.pdf

https://papers.nips.cc/paper/4522-practical-bayesian-optimization-of-machine-learning-algorithms.pdf

https://papers.nips.cc/paper/4443-algorithms-for-hyper-parameter-optimization.pdf

#### Parameters

**iterations:** The number of iterations to perform.

Iteration * functionEvaluationsPerIteration = totalFunctionEvaluations**randomStartingPointCount:** Number of randomly parameter sets used

for initialization (default is 20)**functionEvaluationsPerIterationCount:** The number of function evaluations per iteration.

The parameter sets are included in order of most promising outcome (default is 1)**randomSearchPointCount:** The number of random parameter sets

used when maximizing the expected improvement acquisition function (default is 1000)**seed:****runParallel:** Use multi threading to speed up execution (default is false).

Note that the order of results returned the Optimize method will not be reproducible when running in parallel.

Results will be the same, only the order is not reproducible**maxDegreeOfParallelism: **Maximum number of concurrent operations (default is -1 (unlimited))

## SMAC: Sequential Model-Based Optimization for General Algorithm Configuration

Implementation of the SMAC algorithm.

Based on: Sequential Model-Based Optimization for General Algorithm Configuration:

https://ml.informatik.uni-freiburg.de/papers/11-LION5-SMAC.pdf

And ML.Net implementation:

https://github.com/dotnet/machinelearning/blob/master/src/Microsoft.ML.Sweeper/Algorithms/SmacSweeper.cs

Uses Bayesian optimization in tandem with a greedy local search on the top performing solutions.

#### Parameters

**iterations:** The number of iterations to perform. Iteration * functionEvaluationsPerIteration = totalFunctionEvaluations**randomStartingPointCount: **Number of randomly parameter sets used for initialization (default is 20)**functionEvaluationsPerIterationCount: **The number of function evaluations per iteration.

The parameter sets are included in order of most promising outcome (default is 1)**localSearchPointCount: **The number of top contenders

to use in the greedy local search (default is (10)**randomSearchPointCount: **The number of random parameter sets

used when maximizing the expected improvement acquisition function (default is 1000)**epsilon:** Threshold for ending local search (default is 0.00001)

seed: