How to specify bounds#

Constraints vs bounds#

Estimagic distinguishes between bounds and constraints. Bounds are lower and upper bounds for parameters. In the literature, they are sometimes called box constraints. Examples for general constraints are linear constraints, probability constraints, or nonlinear constraints. You can find out more about general constraints in the next section on How to specify constraints.

Example criterion function#

Let’s again look at the sphere function:

import estimagic as em
import numpy as np

def criterion(x):
    return x @ x

res = em.minimize(criterion, params=np.arange(3), algorithm="scipy_lbfgsb")
res.params

array([ 0.00000000e+00, -1.33177532e-08,  7.18836657e-09])

Array params#

For params that are a numpy.ndarray, one can specify the lower and/or upper-bounds as an array of the same length.

Lower bounds

res = em.minimize(
    criterion, params=np.arange(3), lower_bounds=np.ones(3), algorithm="scipy_lbfgsb"
)
res.params

array([1., 1., 1.])

Lower & upper-bounds

res = em.minimize(
    criterion,
    params=np.arange(3),
    algorithm="scipy_lbfgsb",
    lower_bounds=np.array([-2, -np.inf, 1]),
    upper_bounds=np.array([-1, np.inf, np.inf]),
)
res.params

array([-1.00000000e+00, -3.57647466e-08,  1.00000000e+00])

Pytree params#

Now let’s look at a case where params is a more general pytree. We also update the sphere function by adding an intercept. Since the criterion always decreases when decreasing the intercept, there is no unrestricted solution. Lets fix a lower bound only for the intercept.

params = {"x": np.arange(3), "intercept": 3}


def criterion(params):
    return params["x"] @ params["x"] + params["intercept"]

res = em.minimize(
    criterion,
    params=params,
    algorithm="scipy_lbfgsb",
    lower_bounds={"intercept": -2},
)
res.params

{'x': array([ 0.00000000e+00, -4.42924006e-09,  2.04860640e-08]),
 'intercept': -2.0}

estimagic tries to match the user provided bounds with the structure of params. This allows you to specify bounds for subtrees of params. In case your subtree specification results in an unidentified matching, estimagic will tell you so with a InvalidBoundsError.

params data frame#

It often makes sense to specify your parameters in a pandas.DataFrame, where you can utilize the multiindex for parameter naming. In this case, you can specify bounds as extra columns lower_bound and upper_bound.

Note The columns are called *_bound instead of *_bounds like the argument passed to minimize or maximize.

import pandas as pd

params = pd.DataFrame(
    {"value": [0, 1, 2, 3], "lower_bound": [0, 1, 1, -2]},
    index=pd.MultiIndex.from_tuples([("x", k) for k in range(3)] + [("intercept", 0)]),
)
params
value lower_bound
x 0 0 0
1 1 1
2 2 1
intercept 0 3 -2 
def criterion(params):
    value = (
        params.loc["x"]["value"] @ params.loc["x"]["value"]
        + params.loc["intercept"]["value"]
    )
    return float(value)  # necessary since value is a pd.Series

res = em.minimize(
    criterion,
    params=params,
    algorithm="scipy_lbfgsb",
)
res.params

/tmp/ipykernel_3856/313144487.py:6: FutureWarning: Calling float on a single element Series is deprecated and will raise a TypeError in the future. Use float(ser.iloc[0]) instead
  return float(value)  # necessary since value is a pd.Series
/tmp/ipykernel_3856/313144487.py:6: FutureWarning: Calling float on a single element Series is deprecated and will raise a TypeError in the future. Use float(ser.iloc[0]) instead
  return float(value)  # necessary since value is a pd.Series
/tmp/ipykernel_3856/313144487.py:6: FutureWarning: Calling float on a single element Series is deprecated and will raise a TypeError in the future. Use float(ser.iloc[0]) instead
  return float(value)  # necessary since value is a pd.Series
/tmp/ipykernel_3856/313144487.py:6: FutureWarning: Calling float on a single element Series is deprecated and will raise a TypeError in the future. Use float(ser.iloc[0]) instead
  return float(value)  # necessary since value is a pd.Series
/tmp/ipykernel_3856/313144487.py:6: FutureWarning: Calling float on a single element Series is deprecated and will raise a TypeError in the future. Use float(ser.iloc[0]) instead
  return float(value)  # necessary since value is a pd.Series
value lower_bound
x 0 0.0 0
1 1.0 1
2 1.0 1
intercept 0 -2.0 -2