Optunity API¶
Optunity¶
 Provides
 Routines to efficiently optimize hyperparameters
 Function decorators to implicitly log evaluations, constrain the domain and more.
 Facilities for kfold crossvalidation to estimate generalization performance.
Available modules¶
 solvers
 contains all officially supported solvers
 functions
 a variety of useful function decorators for hyperparameter tuning
 cross_validation
 kfold crossvalidation
Available subpackages¶
 tests
 regression test suite
 solvers
 solver implementations and auxillary functions
Utilities¶
 __version__
 Optunity version string
 __revision__
 Optunity revision string
 __author__
 Main authors of the package

optunity.
manual
(solver_name=None)[source]¶ Prints the manual of requested solver.
Parameters: solver_name – (optional) name of the solver to request a manual from. If none is specified, a general manual is printed. Raises
KeyError
ifsolver_name
is not registered.

optunity.
maximize
(f, num_evals=50, solver_name=None, pmap=<builtin function map>, **kwargs)[source]¶ Basic function maximization routine. Maximizes
f
within the given box constraints.Parameters:  f – the function to be maximized
 num_evals – number of permitted function evaluations
 solver_name (string) – name of the solver to use (optional)
 pmap (callable) – the map function to use
 kwargs – box constraints, a dict of the following form
{'parameter_name': [lower_bound, upper_bound], ...}
Returns: retrieved maximum, extra information and solver info
This function will implicitly choose an appropriate solver and its initialization based on
num_evals
and the box constraints.

optunity.
minimize
(f, num_evals=50, solver_name=None, pmap=<builtin function map>, **kwargs)[source]¶ Basic function minimization routine. Minimizes
f
within the given box constraints.Parameters:  f – the function to be minimized
 num_evals – number of permitted function evaluations
 solver_name (string) – name of the solver to use (optional)
 pmap (callable) – the map function to use
 kwargs – box constraints, a dict of the following form
{'parameter_name': [lower_bound, upper_bound], ...}
Returns: retrieved minimum, extra information and solver info
This function will implicitly choose an appropriate solver and its initialization based on
num_evals
and the box constraints.

optunity.
optimize
(solver, func, maximize=True, max_evals=0, pmap=<builtin function map>, decoder=None)[source]¶ Optimizes func with given solver.
Parameters:  solver – the solver to be used, for instance a result from
optunity.make_solver()
 func (callable) – the objective function
 maximize (bool) – maximize or minimize?
 max_evals (int) – maximum number of permitted function evaluations
 pmap (function) – the map() function to use, to vectorize use
optunity.pmap()
Returns the solution and a
namedtuple
with further details.Result details includes the following:
 optimum
 optimal function value f(solution)
 stats
 statistics about the solving process
 call_log
 the call log
 report
 solver report, can be None
Statistics gathered while solving a problem:
 num_evals
 number of function evaluations
 time
 wall clock time needed to solve
 solver – the solver to be used, for instance a result from

optunity.
wrap_call_log
(f, call_dict)[source]¶ Wraps an existing call log (as dictionary) around f.
This allows you to communicate known function values to solvers. (currently available solvers do not use this info)

optunity.
wrap_constraints
(f, default=None, ub_o=None, ub_c=None, lb_o=None, lb_c=None, range_oo=None, range_co=None, range_oc=None, range_cc=None, custom=None)[source]¶ Decorates f with given input domain constraints.
Parameters:  f (callable) – the function that will be constrained
 default (number) – function value to default to in case of constraint violations
 ub_o (dict) – open upper bound constraints, e.g. \(x < c\)
 ub_c (dict) – closed upper bound constraints, e.g. \(x \leq c\)
 lb_o (dict) – open lower bound constraints, e.g. \(x > c\)
 lb_c (dict) – closed lower bound constraints, e.g. \(x \geq c\)
 range_oo (dict with 2element lists as values ([lb, ub])) – range constraints (open lb and open ub) \(lb < x < ub\)
 range_co (dict with 2element lists as values ([lb, ub])) – range constraints (closed lb and open ub) \(lb \leq x < ub\)
 range_oc (dict with 2element lists as values ([lb, ub])) – range constraints (open lb and closed ub) \(lb < x \leq ub\)
 range_cc (dict with 2element lists as values ([lb, ub])) – range constraints (closed lb and closed ub) \(lb \leq x \leq ub\)
 custom (list of constraints) – custom, userdefined constraints
*custom constraints are binary functions that yield False in case of violations.
>>> def f(x): ... return x >>> fc = wrap_constraints(f, default=1, range_oc={'x': [0, 1]}) >>> fc(x=0.5) 0.5 >>> fc(x=1) 1 >>> fc(x=5) 1 >>> fc(x=0) 1
We can define any custom constraint that we want. For instance, assume we have a binary function with arguments x and y, and we want to make sure that the provided values remain within the unit circle.
>>> def f(x, y): ... return x + y >>> circle_constraint = lambda x, y: (x ** 2 + y ** 2) <= 1 >>> fc = wrap_constraints(f, default=1234, custom=[circle_constraint]) >>> fc(0.0, 0.0) 0.0 >>> fc(1.0, 0.0) 1.0 >>> fc(0.5, 0.5) 1.0 >>> fc(1, 0.5) 1234

optunity.
make_solver
(solver_name, *args, **kwargs)[source]¶ Creates a Solver from given parameters.
Parameters:  solver_name (string) – the solver to instantiate
 args – positional arguments to solver constructor.
 kwargs – keyword arguments to solver constructor.
Use
optunity.manual()
to get a list of registered solvers. For constructor arguments per solver, please refer to Solver overview.Raises
KeyError
ifsolver_name
is not registered*args
and**kwargs
are invalid to instantiate the solver.

optunity.
cross_validated
(x, num_folds=10, y=None, strata=None, folds=None, num_iter=1, regenerate_folds=False, clusters=None, aggregator=<function mean>)[source]¶ Function decorator to perform crossvalidation as configured.
Parameters:  x – data to be used for crossvalidation
 num_folds – number of crossvalidation folds (default 10)
 y – (optional) labels to be used for crossvalidation. If specified, len(labels) must equal len(x)
 strata – (optional) strata to account for when generating folds. Strata signify instances that must be spread across folds. Not every instance must be in a stratum. Specify strata as a list of lists of instance indices.
 folds – (optional) prespecified crossvalidation folds to be used (list of lists (iterations) of lists (folds)).
 num_iter – (optional) number of iterations to use (default 1)
 regenerate_folds – (optional) whether or not to regenerate folds on every evaluation (default false)
 clusters – (optional) clusters to account for when generating folds. Clusters signify instances that must be assigned to the same fold. Not every instance must be in a cluster. Specify clusters as a list of lists of instance indices.
 aggregator – function to aggregate scores of different folds (default: mean)
Returns: a
cross_validated_callable
with the proper configuration.This resulting decorator must be used on a function with the following signature (+ potential other arguments):
Parameters:  x_train (iterable) – training data
 y_train (iterable) – training labels (optional)
 x_test (iterable) – testing data
 y_test (iterable) – testing labels (optional)
y_train and y_test must be available of the y argument to this function is not None.
These 4 keyword arguments will be bound upon decoration. Further arguments will remain free (e.g. hyperparameter names).
>>> data = list(range(5)) >>> @cross_validated(x=data, num_folds=5, folds=[[[i] for i in range(5)]], aggregator=identity) ... def f(x_train, x_test, a): ... return x_test[0] + a >>> f(a=1) [1, 2, 3, 4, 5] >>> f(1) [1, 2, 3, 4, 5] >>> f(a=2) [2, 3, 4, 5, 6]
The number of folds must be less than or equal to the size of the data.
>>> data = list(range(5)) >>> @cross_validated(x=data, num_folds=6) ... def f(x_train, x_test, a): ... return x_test[0] + a AssertionError
The number of labels (if specified) must match the number of data instances.
>>> data = list(range(5)) >>> labels = list(range(3)) >>> @cross_validated(x=data, y=labels, num_folds=2) ... def f(x_train, x_test, a): ... return x_test[0] + a AssertionError

optunity.
generate_folds
(num_rows, num_folds=10, strata=None, clusters=None)[source]¶ Generates folds for a given number of rows.
Parameters:  num_rows – number of data instances
 num_folds – number of folds to use (default 10)
 strata – (optional) list of lists to indicate different sampling strata. Not all rows must be in a stratum. The number of rows per stratum must be larger than or equal to num_folds.
 clusters – (optional) list of lists indicating clustered instances. Clustered instances must be placed in a single fold to avoid information leaks.
Returns: a list of folds, each fold is a list of instance indices
>>> folds = generate_folds(num_rows=6, num_folds=2, clusters=[[1, 2]], strata=[[3,4]]) >>> len(folds) 2 >>> i1 = [idx for idx, fold in enumerate(folds) if 1 in fold] >>> i2 = [idx for idx, fold in enumerate(folds) if 2 in fold] >>> i1 == i2 True >>> i3 = [idx for idx, fold in enumerate(folds) if 3 in fold] >>> i4 = [idx for idx, fold in enumerate(folds) if 4 in fold] >>> i3 == i4 False
Warning
Instances in strata are not necessarily spread out over all folds. Some folds may already be full due to clusters. This effect should be negligible.

optunity.
pmap
(f, *args, **kwargs)[source]¶ Parallel map using multiprocessing.
Parameters:  f – the callable
 args – arguments to f, as iterables
Returns: a list containing the results
Warning
This function will not work in IPython: https://github.com/claesenm/optunity/issues/8.
Warning
Python’s multiprocessing library is incompatible with Jython.

optunity.
available_solvers
()[source]¶ Returns a list of all available solvers.
These can be used in
optunity.make_solver()
.

optunity.
call_log2dataframe
(log)[source]¶ Converts a call log into a pandas data frame. This function errors if you don’t have pandas available.
Parameters: log – call log to be converted, as returned by e.g. optunity.minimize Returns: a pandas data frame capturing the same information as the call log

optunity.
maximize_structured
(f, search_space, num_evals=50, pmap=<builtin function map>)[source]¶ Basic function maximization routine. Maximizes
f
within the given box constraints.Parameters:  f – the function to be maximized
 search_space – the search space (see Structured search spaces for details)
 num_evals – number of permitted function evaluations
 pmap (callable) – the map function to use
Returns: retrieved maximum, extra information and solver info
This function will implicitly choose an appropriate solver and its initialization based on
num_evals
and the box constraints.

optunity.
minimize_structured
(f, search_space, num_evals=50, pmap=<builtin function map>)[source]¶ Basic function minimization routine. Minimizes
f
within the given box constraints.Parameters:  f – the function to be maximized
 search_space – the search space (see Structured search spaces for details)
 num_evals – number of permitted function evaluations
 pmap (callable) – the map function to use
Returns: retrieved maximum, extra information and solver info
This function will implicitly choose an appropriate solver and its initialization based on
num_evals
and the box constraints.