{ "cells": [ { "cell_type": "markdown", "id": "55b3f1ad", "metadata": {}, "source": [ "# How to specify bounds\n", "\n", "## Constraints vs bounds \n", "\n", "Estimagic distinguishes between bounds and constraints. Bounds are lower and upper bounds for parameters. In the literature, they are sometimes called box constraints. Examples for general constraints are linear constraints, probability constraints, or nonlinear constraints. You can find out more about general constraints in the next section on [How to specify constraints](how_to_specify_constraints.md)." ] }, { "cell_type": "markdown", "id": "b3c135aa", "metadata": {}, "source": [ "## Example criterion function\n", "\n", "Let’s again look at the sphere function:" ] }, { "cell_type": "code", "execution_count": 1, "id": "ec477eb7", "metadata": {}, "outputs": [], "source": [ "import estimagic as em\n", "import numpy as np" ] }, { "cell_type": "code", "execution_count": 2, "id": "b0eb906d", "metadata": {}, "outputs": [], "source": [ "def criterion(x):\n", " return x @ x" ] }, { "cell_type": "code", "execution_count": 3, "id": "6b43b46e", "metadata": {}, "outputs": [ { "data": { "text/plain": [ "array([ 0.00000000e+00, -1.33177532e-08, 7.18836657e-09])" ] }, "execution_count": 3, "metadata": {}, "output_type": "execute_result" } ], "source": [ "res = em.minimize(criterion, params=np.arange(3), algorithm=\"scipy_lbfgsb\")\n", "res.params" ] }, { "cell_type": "markdown", "id": "069788ac", "metadata": {}, "source": [ "## Array params\n", "\n", "For params that are a `numpy.ndarray`, one can specify the lower and/or upper-bounds as an array of the same length.\n", "\n", "**Lower bounds**" ] }, { "cell_type": "code", "execution_count": 4, "id": "0c450bdd", "metadata": {}, "outputs": [ { "data": { "text/plain": [ "array([1., 1., 1.])" ] }, "execution_count": 4, "metadata": {}, "output_type": "execute_result" } ], "source": [ "res = em.minimize(\n", " criterion, params=np.arange(3), lower_bounds=np.ones(3), algorithm=\"scipy_lbfgsb\"\n", ")\n", "res.params" ] }, { "cell_type": "markdown", "id": "737501d6", "metadata": {}, "source": [ "**Lower & upper-bounds**" ] }, { "cell_type": "code", "execution_count": 5, "id": "26c5c0df", "metadata": {}, "outputs": [ { "data": { "text/plain": [ "array([-1.00000000e+00, -3.57647466e-08, 1.00000000e+00])" ] }, "execution_count": 5, "metadata": {}, "output_type": "execute_result" } ], "source": [ "res = em.minimize(\n", " criterion,\n", " params=np.arange(3),\n", " algorithm=\"scipy_lbfgsb\",\n", " lower_bounds=np.array([-2, -np.inf, 1]),\n", " upper_bounds=np.array([-1, np.inf, np.inf]),\n", ")\n", "res.params" ] }, { "cell_type": "markdown", "id": "95065b2b", "metadata": {}, "source": [ "## Pytree params\n", "\n", "Now let's look at a case where params is a more general pytree. We also update the sphere function by adding an intercept. Since the criterion always decreases when decreasing the intercept, there is no unrestricted solution. Lets fix a lower bound only for the intercept." ] }, { "cell_type": "code", "execution_count": 6, "id": "9c05eb78", "metadata": {}, "outputs": [], "source": [ "params = {\"x\": np.arange(3), \"intercept\": 3}\n", "\n", "\n", "def criterion(params):\n", " return params[\"x\"] @ params[\"x\"] + params[\"intercept\"]" ] }, { "cell_type": "code", "execution_count": 7, "id": "ddcc54d4", "metadata": {}, "outputs": [ { "data": { "text/plain": [ "{'x': array([ 0.00000000e+00, -4.42924006e-09, 2.04860640e-08]),\n", " 'intercept': -2.0}" ] }, "execution_count": 7, "metadata": {}, "output_type": "execute_result" } ], "source": [ "res = em.minimize(\n", " criterion,\n", " params=params,\n", " algorithm=\"scipy_lbfgsb\",\n", " lower_bounds={\"intercept\": -2},\n", ")\n", "res.params" ] }, { "cell_type": "markdown", "id": "096d3ba4", "metadata": {}, "source": [ "estimagic tries to match the user provided bounds with the structure of params. This allows you to specify bounds for subtrees of params. In case your subtree specification results in an unidentified matching, estimagic will tell you so with a `InvalidBoundsError`. " ] }, { "cell_type": "markdown", "id": "1b7302c1", "metadata": {}, "source": [ "## params data frame\n", "\n", "It often makes sense to specify your parameters in a `pandas.DataFrame`, where you can utilize the multiindex for parameter naming. In this case, you can specify bounds as extra columns `lower_bound` and `upper_bound`.\n", "\n", "> **Note**\n", "> The columns are called `*_bound` instead of `*_bounds` like the argument passed to `minimize` or `maximize`. " ] }, { "cell_type": "code", "execution_count": 8, "id": "b4a95453", "metadata": {}, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
valuelower_bound
x000
111
221
intercept03-2
\n", "
" ], "text/plain": [ " value lower_bound\n", "x 0 0 0\n", " 1 1 1\n", " 2 2 1\n", "intercept 0 3 -2" ] }, "execution_count": 8, "metadata": {}, "output_type": "execute_result" } ], "source": [ "import pandas as pd\n", "\n", "params = pd.DataFrame(\n", " {\"value\": [0, 1, 2, 3], \"lower_bound\": [0, 1, 1, -2]},\n", " index=pd.MultiIndex.from_tuples([(\"x\", k) for k in range(3)] + [(\"intercept\", 0)]),\n", ")\n", "params" ] }, { "cell_type": "code", "execution_count": 9, "id": "34d59f01", "metadata": {}, "outputs": [], "source": [ "def criterion(params):\n", " value = (\n", " params.loc[\"x\"][\"value\"] @ params.loc[\"x\"][\"value\"]\n", " + params.loc[\"intercept\"][\"value\"]\n", " )\n", " return float(value) # necessary since value is a pd.Series" ] }, { "cell_type": "code", "execution_count": 10, "id": "b284ad8a", "metadata": {}, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
valuelower_bound
x00.00
11.01
21.01
intercept0-2.0-2
\n", "
" ], "text/plain": [ " value lower_bound\n", "x 0 0.0 0\n", " 1 1.0 1\n", " 2 1.0 1\n", "intercept 0 -2.0 -2" ] }, "execution_count": 10, "metadata": {}, "output_type": "execute_result" } ], "source": [ "res = em.minimize(\n", " criterion,\n", " params=params,\n", " algorithm=\"scipy_lbfgsb\",\n", ")\n", "res.params" ] } ], "metadata": { "kernelspec": { "display_name": "Python 3 (ipykernel)", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.10.8" } }, "nbformat": 4, "nbformat_minor": 5 }