Structured search spaces

Functionality to deal with structured search spaces, in which the existence of some hyperparameters is contingent on some discrete choice(s).

For more information on the syntax and modalities of defining structured search spaces, please refer to Structured search spaces.

A search space is defined as a dictionary mapping strings to nodes. Each node is one of the following:

  1. A new sub search space, that is a new dictionary with the same structure.
  2. 2-element list or tuple, containing (lb, ub) for the associated hyperparameter.
  3. None, to indicate a terminal node that has no numeric value associated to it.

Internally, structured search spaces are encoded as a vector, based on the following rules:

  • a standard hyperparameter with box constraints is a single dimension
  • when a choice must be maded, this is encoded as a single dimension (regardless of the amount of options), with values in the range [0, num_choices]
  • choice options without additional hyperparameters (i.e., None nodes) do not require further coding

For example, consider an SVM kernel family, choosing between linear and RBF. This is specified as follows:

search = {'kernel': {'linear': None,
                     'rbf': {'gamma': [0, 3]}
                    }
         }

In vector format, this is encoded as a vector with 2 entries \([kernel \in [0, 2],\ gamma \in [0, 3]]\).

The main API of this module is the SearchTree class.

Module author: Marc Claesen

class optunity.search_spaces.Node(key, value)[source]

Bases: object

Models a node within a search space.

Nodes can be internal or terminal, and may or may not be choices. A choice is a node that models a discrete choice out of k > 1 options.

choice

Determines whether this node is a choice.

A choice is a node that models a discrete choice out of k > 1 options.

key
terminal

Returns whether or not this Node is terminal.

A terminal node has a non-dictionary value (numeric, list or None).

value
class optunity.search_spaces.Options(cases)[source]

Bases: object

cases
class optunity.search_spaces.SearchTree(d)[source]

Bases: object

Tree structure to model a search space.

Fairly elaborate unit test.

>>> space = {'a': {'b0': {'c0': {'d0': {'e0': [0, 10], 'e1': [-2, -1]},
...                              'd1': {'e2': [-3, -1]},
...                              'd2': None
...                              },
...                       'c1': [0.0, 1.0],
...                      },
...                'b1': {'c2': [-2.0, -1.0]},
...                'b2': None
...               }
...          }
>>> tree = SearchTree(space)
>>> b = tree.to_box()
>>> print(b['a'] == [0.0, 3.0] and
...       b['a|b0|c0'] == [0.0, 3.0] and
...       b['a|b0|c1'] == [0.0, 1.0] and
...       b['a|b1|c2'] == [-2.0, -1.0] and
...       b['a|b0|c0|d0|e0'] == [0, 10] and
...       b['a|b0|c0|d0|e1'] == [-2, -1] and
...       b['a|b0|c0|d1|e2'] == [-3, -1])
True
>>> d = tree.decode({'a': 2.5})
>>> d['a'] == 'b2'
True
>>> d = tree.decode({'a': 1.5, 'a|b1|c2': -1.5})
>>> print(d['a'] == 'b1' and
...       d['c2'] == -1.5)
True
>>> d = tree.decode({'a': 0.5, 'a|b0|c0': 1.7, 'a|b0|c0|d1|e2': -1.2})
>>> print(d['a'] == 'b0' and
...       d['c0'] == 'd1' and
...       d['e2'] == -1.2)
True
>>> d = tree.decode({'a': 0.5, 'a|b0|c0': 2.7})
>>> print(d['a'] == 'b0' and
...       d['c0'] == 'd2')
True
>>> d = tree.decode({'a': 0.5, 'a|b0|c0': 0.7, 'a|b0|c0|d0|e0': 2.3, 'a|b0|c0|d0|e1': -1.5})
>>> print(d['a'] == 'b0' and
...       d['c0'] == 'd0' and
...       d['e0'] == 2.3 and
...       d['e1'] == -1.5)
True
content
decode(vd)[source]

Decodes a vector representation (as a dictionary) into a result dictionary.

Parameters:vd (dict) – vector representation
Returns:dict containing the decoded representation.
  • Choices are given as key:value pairs.
  • Active hyperparameters have numeric values.
  • Inactive hyperparameters have value None.
to_box()[source]

Creates a set of box constraints to define the given search space.

Returns:a set of box constraints (e.g. dictionary, with box constraint values)

Use these box constraints to initialize a solver, which will then implicitly work in the vector representation of the search space.

To evaluate such vector representations, decorate the objective function with optunity.search_spaces.SearchTree.decode() of the same object.

vectorcontent
vectordict
wrap_decoder(f)[source]

Wraps a function to automatically decode arguments based on given SearchTree.

Use in conjunction with optunity.search_spaces.SearchTree.to_box().