Enforcing constraints by reparametrizations and bounds

Handle constraints by reparametrizations.

The functions in this module allow to convert between internal and external parameter vectors.

An external parameter vector is the parameter vector as it was specified by the user. This external parameter vector might be subject to constraints, such as the condition that the first two parameters are equal.

An internal parameter vector is an internal representation of the parameters in a different space. The internal parameters are meaningless and have no direct interpretation. However, the internal parameter vector has two important properties: 1. It is only subject to box constraints 2. reparametrize_from_internal(internal_parameter) always produces a valid external parameter vector (i.e. one that fulfills all constraints.

For more background see How constraints are implemented.

The reparametrization from internal can be broken down into three separate steps:

• Writing values from the internal parameter vector into an array that is as long as the external parameters and contains NaNs or values to which parameters have been fixed. We call this step pre_replace.

• Transforming slices of the resulting vector with kernel transformations. Note that this step does not change the length. All kernel transformations have as many input as output parameters and are invertible. We call this step transformation. The resulting vector might still contrain NaNs.

• Fill the NaNs by duplicating values of the transformed parameter vector. We call this step post_replace

In the following, let n_external be the length of th external parameter vector and n_internal the length of the internal parameter vector.

estimagic.optimization.reparametrize.reparametrize_to_internal(external, internal_free, processed_constraints)

Convert a params DataFrame into a numpy array of internal parameters.

Parameters

• external (np.ndarray) – 1d array with external parameters.

• internal_free (np.ndarray) – 1d array of lenth n_external that determines which parameters are free.

• processed_constraints (list) – Processed and consolidated constraints. The processed constraints contain information on the transformations that have to be done.

Returns

1d numpy array of free reparametrized

parameters.

Return type

internal_params (numpy.ndarray)

estimagic.optimization.reparametrize.reparametrize_from_internal(internal, fixed_values, pre_replacements, processed_constraints, post_replacements)

Convert a numpy array of internal parameters to a params DataFrame.

Parameters

• internal (numpy.ndarray) – 1d numpy array with internal parameters

• fixed_values (numpy.ndarray) – 1d numpy array of length n_external. It contains NaN for parameters that are not fixed and an internal representation of the value to which a parameter has been fixed for all others.

• pre_replacements (numpy.ndarray) – 1d numpy of length n_external. The i_th element in array contains the position of the internal parameter that has to be copied to the i_th position of the external parameter vector or -1 if no value has to be copied.

• processed_constraints (list) – List of processed and consolidated constraint dictionaries. Can have the types “linear”, “probability”, “covariance” and “sdcorr”.

• post_replacements (numpy.ndarray) – 1d numpy array of lenth n_external. The i_th element contains the position a parameter in the transformed parameter vector that has to be copied to duplicated and copied to the i_th position of the external parameter vector.

Returns

Array with external parameters

Return type

numpy.ndarray

estimagic.optimization.reparametrize.convert_external_derivative_to_internal(external_derivative, internal_values, fixed_values, pre_replacements, processed_constraints, post_replacements=None, pre_replace_jac=None, post_replace_jac=None)

Compute the derivative of the criterion utilizing an external derivative.

Denote by \(c\) the criterion function which is evaluated on the full parameter set. Denote by \(g\) the paramater transform which maps an internal to an external paramter, i.e \(g: x \mapsto g(x)\), with \(x\) denoting the internal paramter vector and \(g(x)\) the respective external parameter frame. We are interested in the derivative of the composition \(f := c \circ g\) which maps an internal vector to the criterion value. The derivative can be computed using the chain rule, as

\[\frac{\mathrm{d}f}{\mathrm{d}x}(x) = \frac{\mathrm{d}c}{\mathrm{d}g}(g(x)) \times \frac{\mathrm{d}g}{\mathrm{d}x}(x)\]

We assume that the user provides the first part of the above product. The second part denotes the derivative of the paramter transform from inner to external.

Parameters

• external_derivative (numpy.ndarray) – The external derivative evaluated at external values mapped from internal_values.

• internal_values (numpy.ndarray) – 1d numpy array with internal parameters

• fixed_values (numpy.ndarray) – 1d numpy array of length n_external. It contains NaN for parameters that are not fixed and an internal representation of the value to which a parameter has been fixed for all others.

• pre_replacements (numpy.ndarray) – 1d numpy of length n_external. The i_th element in array contains the position of the internal parameter that has to be copied to the i_th position of the external parameter vector or -1 if no value has to be copied.

• processed_constraints (list) – List of processed and consolidated constraint dictionaries. Can have the types “linear”, “probability”, “covariance” and “sdcorr”.

• post_replacements (numpy.ndarray) – 1d numpy array of lenth n_external. The i_th element contains the position a parameter in the transformed parameter vector that has to be copied to duplicated and copied to the i_th position of the external parameter vector.

• pre_replace_jac (np.ndarray) – 2d Array with the jacobian of pre_replace

• post_replacment_jacobian (np.ndarray) – 2d Array with the jacobian post_replace

Returns

The gradient or Jacobian.

Return type

deriv (numpy.ndarray)

estimagic.optimization.reparametrize.pre_replace(internal_values, fixed_values, pre_replacements)

Return pre-replaced parameters.

Parameters

• internal (numpy.ndarray) – 1d numpy array with internal parameter.

• fixed_values (numpy.ndarray) – 1d numpy array of length n_external. It contains NaN for parameters that are not fixed and an internal representation of the value to which a parameter has been fixed for all others.

• pre_replacements (numpy.ndarray) – 1d numpy of length n_external. The i_th element in array contains the position of the internal parameter that has to be copied to the i_th position of the external parameter vector or -1 if no value has to be copied.

Returns

1d numpy array with pre-replaced params.

Return type

pre_replaced (numpy.ndarray)

Examples

>>> internal_values = np.array([1., 2.])
>>> fixed_values = np.array([np.nan, 0, np.nan])
>>> pre_replacements = np.array([1, -1, 0])
>>> pre_replace(internal_values, fixed_values, pre_replacements)
array([2., 0., 1.])

estimagic.optimization.reparametrize.pre_replace_jacobian(pre_replacements, dim_in)

Return Jacobian of pre-replacement step.

Remark. The function pre_replace can have np.nan in its output. In this case we know from the underlying structure that the derivative of this output with respect to any of the inputs is zero. Here we use this additional knowledge; however, when the derivative is computed using a numerical differentiation technique this will not be the case. Thus the numerical derivative can differ from the derivative here in these cases.

Parameters

• pre_replacements (numpy.ndarray) – 1d numpy of length n_external. The i_th element in array contains the position of the internal parameter that has to be copied to the i_th position of the external parameter vector or -1 if no value has to be copied.

• dim_in (int) – Dimension of the internal parameters.

Returns

The jacobian.

Return type

jacobian (np.ndarray)

Examples

>>> # Note: The example is the same as in the doctest of pre_replace
>>> pre_replacements = np.array([1, -1, 0])
>>> pre_replace_jacobian(pre_replacements, 2)
array([[0., 1.],
       [0., 0.],
       [1., 0.]])

estimagic.optimization.reparametrize.transformation_jacobian(processed_constraints, pre_replaced)

Return Jacobian of constraint transformation step.

The Jacobian of the constraint transformation step is build as a block matrix of either identity matrices, in the case when the external parameter equals the internal parameter, or, of the Jacobians of the specific kernel transforms, in case the external paramater is a transformed version of the internal.

Parameters

• processed_constraints (list) – List of processed and consolidated constraint dictionaries. Can have the types “linear”, “probability”, “covariance” and “sdcorr”.

• pre_replaced (numpy.ndarray) – 1d numpy array with pre-replaced params.

• dim (int) – The dimension of the external parameters.

Returns

The Jacobian.

Return type

jacobian (numpy.ndarray)

estimagic.optimization.reparametrize.post_replace(external_values, post_replacements)

Return post-replaed parameters.

Parameters

• external_values (numpy.ndarray) – 1d numpy array of external params.

• post_replacements (numpy.ndarray) – 1d numpy array of lenth n_external. The i_th element contains the position a parameter in the transformed parameter vector that has to be copied to duplicated and copied to the i_th position of the external parameter vector.

Returns

1d numpy array with post-replaced params.

Return type

post_replaced (numpy.ndarray)

Examples

>>> external_values = np.array([3., 4., np.nan])
>>> post_replacements = np.array([-1, -1, 1])
>>> post_replace(external_values, post_replacements)
array([3., 4., 4.])

estimagic.optimization.reparametrize.post_replace_jacobian(post_replacements)

Return Jacobian of post-replacement step.

Parameters

• post_replacements (numpy.ndarray) – 1d numpy array of lenth n_external. The i_th element contains the position a parameter in the transformed parameter vector that has to be copied to duplicated and copied to the i_th position of the external parameter vector.

• dim (int) – The dimension of the external parameters.

Returns

The Jacobian.

Return type

jacobian (np.ndarray)

Examples

>>> # Note: the example is the same as in the doctest of post_replace
>>> post_replacements = np.array([-1, -1, 1])
>>> post_replace_jacobian(post_replacements)
array([[1., 0., 0.],
       [0., 1., 0.],
       [0., 1., 0.]])

Processing constraints The default algorithm options