Solver overviewΒΆ

We currently recommend using Particle Swarm Optimization (our default). Based on our experience this is the most reliable solver across different learning algorithms. If you consistently get great results with a solver/algorithm combination, we are happy to hear about your experiences.

You can specify which solver you want to use in optunity.maximize() and optunity.minimize(), but only limited configuration is possible. If you want to specify detailed settings for a solver, you can use the expert interface, specifically optunity.make_solver() in combination with optunity.optimize().

The following solvers are available in Optunity:

Optunity’s default solver is Particle Swarm Optimization.

Grid Search, Random Search and Sobol sequences are completely undirected algorithms and consequently not very efficient. Of these three, Sobol sequences is most efficient as uses a low-discrepancy quasirandom sequence.

Nelder-Mead simplex works well for objective functions that are smooth, unimodal and not too noisy (it is good for local search when you have a good idea about optimal regions for your hyperparameters).

For general searches, Particle Swarm Optimization and CMA-ES are most robust. Finally, the Tree-structured Parzen Estimator solver exposes Hyperopt’s TPE solver in Optunity’s familiar API.