ALAMO is designed for situations in which a small, explicit algebraic expression is the desired end product. This is the case when the surrogate is to be embedded in a subsequent optimization model, when it is to be inspected or interpreted by a domain expert, or when it must be transcribed into a highly efficient control system or simulator. ALAMO is well suited to problems in which

• •

  data are expensive, so that the number of function evaluations must be kept small;

• •

  the number of input variables is modest, typically fewer than a few dozen;

• •

  the functional form is not known in advance but is expected to be sparse in a physically meaningful basis; and

• •

  known physics, such as bounds, limits, or boundary conditions, should be imposed on the surrogate.

ALAMO is not the right tool for every modeling task. If a well-established mechanistic model already exists, fitting its parameters may be preferable to learning a new functional form. If the goal is prediction accuracy alone and the model will never be optimized or read by a human, a neural network, a Gaussian process, or a gradient-boosted tree may fit the data more closely. If the goal is to optimize a black box directly, without building a model of it first, a derivative-free optimization solver such as BAM [20] is a more direct route; see https://minlp.com/bam-data-driven-optimizer.

The code is available for download at http://minlp.com/alamo. The same URL provides information about licensing the software. The software is free for academics, but a license is required to run it. ALAMO is stand-alone software, with no third party dependencies.

Install ALAMO and the ALAMO license file in any directory of your choice and add it to your path. On Windows, ALAMO’s installer will take care of these steps for you. On Linux and OSX systems, unzip the ALAMO download and place the files it contains in a directory in your path. Users should not open the license file in an editor as this may invalidate the license in certain operating systems. For all operating systems, make sure that ALAMO and the ALAMO license file are readable by all intended users on your machine.

For a silent (non-interactive) installation on Windows, download the installer and run it as follows from the command line:

    alamo-windows64.exe /SILENT

If an installation log file is additionally desired, it can be generated by the command:

    alamo-windows64.exe /SILENT /LOG=filename

where $\mathrm{𝚏𝚒𝚕𝚎𝚗𝚊𝚖𝚎}$ is the desired name for the log file. In both cases, Windows will still request permission to run the executable before the silent installer is launched.

Consider a system whose outputs $z$ are an unknown function of its inputs $x\in {ℝ}^n$, where $n$ is NINPUTS. ALAMO models each output variable separately, so it suffices to describe the treatment of a single output $z$; when NOUTPUTS exceeds one, the entire procedure of this section is repeated for each output that is not excluded by IGNORE. The data available at any point of the algorithm consist of $N$ measurements

where $x_i=(x_{i1},\mathrm{\dots },x_{in})$ and $z_i\in ℝ$. These measurements may be supplied by the user (NDATA), generated by sampling a simulator (NSAMPLE), or drawn from a data bank (NBANK), and their number grows as adaptive sampling proceeds.

ALAMO restricts its attention to models that are linear in a set of nonlinear transformations of the inputs. Given a collection $ℬ$ of $J=|ℬ|$ candidate basis functions $X_j:{ℝ}^n\to ℝ$, the surrogate model has the form

where ${\beta }_j$ is the regression coefficient of the $j$th basis function. Writing $X_{ij}=X_j(x_i)$ for the design matrix, the ordinary least squares (OLR) estimate of $\beta$ solves

Two properties of (1) are worth emphasizing, because they are what distinguish ALAMO from most surrogate modeling software. First, the model is algebraic and explicit: it can be printed, read, differentiated, and handed to an algebraic optimization solver. Second, the model is a sparse linear combination. The point of the algorithm is not to estimate $\beta$ over all of $ℬ$, which (2) already does, but to decide which of the ${\beta }_j$ should be nonzero at all. Denoting by $𝒮\subseteq ℬ$ the set of selected basis functions and by $p=|𝒮|$ the model size, ALAMO solves a best subset selection problem

where $\mathrm{\Phi }$ is a goodness-of-fit measure that rewards accuracy and penalizes model size. The choice of $\mathrm{\Phi }$ is the MODELER option and is discussed in Section 3.4.1.

Surrogate models are constructed using the three-step process shown in Figure 1 [5].

In the first step, an initial design of experiments is generated and the system is measured at these points. In the second step, an algebraic model is built from this initial training set. The model is built using integer optimization techniques to select the best subset from the collection of potential basis functions. In the third step, an adaptive sampling methodology based on derivative-free optimization techniques [25] is used to identify points where the model is inaccurate. Once these points are added to the training set, execution returns to the second step. The process continues until the third step confirms the accuracy of a previously built model, or until MAXITER, MAXTIME, the work-unit budget MAXWORK of Section 6.10, or the tolerances of Section 6.9 stop it.

Two features of this loop are worth stating explicitly. The model-building step and the sampling step are coupled: each new model determines where the next measurements are taken, and each new measurement changes which basis functions are selected. And the loop is designed for the regime in which measurements are expensive, so it spends optimization effort in order to save function evaluations rather than the reverse.

Compared with common techniques such as forward selection or backward elimination, which investigate model sensitivities with respect to one basis function at a time, ALAMO’s best subset selection accounts for the synergistic effects between different basis functions [32]. Before ALAMO, best subset selection was considered too time consuming for realistic data sets; exhaustive enumeration schemes such as leaps-and-bounds [10] scale poorly, and popular implementations are limited to a few dozen candidate regressors. While developing ALAMO, mixed-integer programming formulations were devised that rely on the BARON software [34, 2] to solve these models in realistic computing times for many industrially relevant systems. ALAMO is also unique in that it utilizes derivative-free optimization techniques in its adaptive sampling step, and in its constrained regression feature, which is capable of enforcing theory-driven requirements on the response variable [6].

Everything ALAMO can produce is a subset of what it is offered. The candidate set $ℬ$ is therefore the single most consequential input to the algorithm, and the options of Section 6.6 exist to define it.

The simple basis functions are of the categories shown in Table 1. They are chosen from first-principles relationships, physical or engineering insight, or standard statistical fitting functions. The exponents $\alpha$ and the scaling constant $\gamma$ are supplied by the user; values of $\alpha$ such as $\pm \frac{1}{2},\pm 1,\pm 2,\pm 3,\pm 4$ and of $\gamma$ such as $0.1,1,10$ produce either physically reasonable basis functions or common statistical fitting functions.

The virtue of a diverse candidate set is that it can represent a wide variety of unknown functional forms even when none of its members matches the truth exactly. For example, if $z(x)=2x^{3/2}+1$ is modeled over $x\in [0,1]$, a purely quadratic surrogate $\widehat{z}(x)=1.3x^2-0.35x+2$ already has an average error of about one percent [5]. Conversely, an over-generous candidate set is expensive: with five levels of $\alpha$ for polynomials and multinomials, pairwise multinomial terms with equal exponents, and $\gamma =1$, a two-, four-, and ten-dimensional problem generates 20, 59, and 246 candidate basis functions, respectively [5]. Since the difficulty of (3) grows combinatorially in $J$, the candidate set should be rich in the directions the physics suggests and thin elsewhere. The EXCLUDE option removes input variables from consideration, and NUMLIMITBASIS removes basis functions that are not numerically well defined over the box defined by XMIN and XMAX, which is what silently eliminates logarithms when an input may be nonpositive.

Custom basis functions deserve particular mention. Any expression the user can write in terms of the XLABELS may be added to $ℬ$, which is how Arrhenius, sigmoidal, and other structured relationships enter the model: the nonlinear parameter inside such a relationship is fixed and plays the role of a hyperparameter, while the coefficient multiplying it is estimated by regression. This is also the mechanism by which a known mechanistic term can be offered to the model alongside the generic bases, so that ALAMO can decide from the data whether it is needed.

A model built from a fixed data set can be assessed only against that data set. Its empirical error is computable, but its true error is not, and no amount of cross validation on the existing points demonstrates that the problem space has been sampled adequately. ALAMO addresses this by interrogating the black box directly, using the current model to decide where to look. The technique is called error maximization sampling (EMS).

Data are not the only information a modeler has. Concentrations and absolute pressures are nonnegative, mole fractions sum to one and lie in $[0,1]$, a heat duty is bounded by the corresponding Carnot limit, an entropy is monotone, and a velocity profile vanishes at a wall. Information of this kind is usually available at no cost, and a surrogate that violates it is not merely inaccurate but unphysical, which matters a great deal when the surrogate is subsequently optimized: an optimizer will find and exploit the region in which the model is unphysical. ALAMO’s constrained regression facility enforces such requirements on the response surface over the entire input domain [6].

The design choices described above have been tested extensively, and the results are worth summarizing, both because they justify the defaults and because they indicate what to expect.

Best subset selection versus alternatives. On a test set of 45 functions built from bases available to every method, the deviation of the surrogate model size from the true model size, $T^{\mathrm{surrogate}}-T^{\mathrm{true}}$, averaged $-0.86$ with a standard deviation of 1.69 for ALAMO’s best subset modeler, against $4.62$ (5.01) for the lasso and $10.18$ (8.26) for ordinary least squares regression [5]. The best subset models are not only smaller; across both of the test sets used in that study they were also the most accurate on independent validation data.

Error maximization sampling versus space-filling designs. On a set of 150 reaction modeling problems, EMS produced a better model than a Latin hypercube design [22] of the same size in 120 of the 150 cases [32]. The number of points EMS required to reach a tolerance of ${10}^{-4}$ ranged from 14 to 46, with a mean of 21.7 and a median of 21, and 97 percent of the problems converged in fewer than 30 samples. Averaging 21 points, EMS attained a mean $R^2$ on validation data of 0.994, against 0.965 for a fixed Latin hypercube design of 20 points.

Constrained regression. On the same 150 problems, extending the enforcement domain beyond the training range produced the most accurate model on the lower extrapolation region in 117 cases, against 29 for constraints enforced only over the training domain and 9 for the unconstrained models [32]. Within the training domain the ordering reverses, as it must, since the unconstrained problem is a relaxation of the constrained one.

Choice of metric. On benchmarks drawn from the UC Irvine repository, the NIST standard regression database, and the Virtual Library of Simulated Experiments, BIC solved the resulting subproblems more than two orders of magnitude faster than AIC, MSE, and HQC, and produced smaller and more accurate models, on account of its heavier penalty on model size and of the direct mixed-integer quadratic formulation (14) it admits. This is the reason BIC is the default value of MODELER.

The bibliography at the end of this document offers more details of the methodology implemented in ALAMO and demonstrates the advantages of this methodology in comparison to currently utilized approaches, including classical regression and the lasso. The methodology of Sections 3.4 and 3.5 is developed in [5] and reviewed and compared against the machine learning literature in [32]; constrained regression is developed in [6]; a comprehensive treatment, including material that has not appeared elsewhere, is given in [4]; and the transfer of information criteria between related modeling problems is treated in [3]. Parts of the algorithm have been implemented commercially in the gPROMS simulator [23].

The ALAMO GUI allows for a convenient spreadsheet-style input of problem data and algorithmic options, and offers facilities for visualizing the data and results, and saving your work. On all operating systems, Java is required in order to use the ALAMO GUI. Additionally, the user must have write privileges in the directory where the GUI is invoked.

On Windows systems, the installer will create an ALAMO icon on your desktop. Double click at this icon in order to launch ALAMO’s interactive GUI.

On Linux and OSX, the interactive GUI is available in the download directory and named alamogui.jar. To run this, open a terminal and type the following command at the prompt

    java -jar alamogui.jar

On all systems, the GUI allows you to save the model input data in the form of an ALAMO file that can be read later by the GUI. After running ALAMO, the results can be saved in an ALAMO listing file that can also be read later by the GUI to reconstruct the problem inputs and results.

Even if you are not planning to use ALAMO from the command line, you should still plan to quickly read the next subsection and following section as they explain how ALAMO works and what is expected to make it run correctly. Additionally, these sections describe material related to algorithmic options that are available in a very similar format through the GUI.

As an alternative to using the GUI, users can utilize ALAMO from the command line. ALAMO reads model data and algorithmic options from a text file in a simple format. Even though it is not required, it is strongly recommended that all ALAMO input files have the extension ‘.alm.’ If the input file is named ‘test.alm’ and the ALAMO executable is named ‘alamo,’ issuing the command

    alamo test

or

    alamo test.alm

results in ALAMO parsing test.alm and solving the problem. In addition to screen displays, ALAMO can also provide results in the listing file ‘test.lst’ that is generated during the run. The .lst file is always stored in the execute directory, even when the .alm file is in a different path. During execution, ALAMO creates and utilizes a directory for storing various work files. When calling ALAMO, the user may optionally include a second command line argument in order to specify ALAMO’s working directory:

    alamo test.alm myscratchdir

where ‘myscratchdir’ denotes the name of ALAMO’s scratch directory. If this argument is not specified, ALAMO will create and utilize a directory named ‘almscr’ in the execute directory. If the scratch exists, it is erased in the beginning of the run. At the end of the run, ALAMO will delete its scratch directory.

In addition to the GUI and the command line, ALAMO can be driven directly from a calling program through a callable library. The library exposes the same functionality as the input file: the caller initializes ALAMO, sets options and data through a collection of set routines, requests a solve, and retrieves the resulting model and statistics through a collection of get routines. No .alm file is involved.

This interface is documented in full, with complete worked examples in C, fixed-form Fortran and Fortran 90, in Section 8.

ALAMO is available to Python users through ALAMOPY (Automatic Learning of Algebraic MOdels PYthon wrapper), which is distributed as part of the IDAES Integrated Platform [18, 30]. ALAMOPY is a wrapper around the ALAMO executable: it writes an input file, invokes ALAMO, and parses the results back into Python objects. A licensed installation of ALAMO is therefore required; ALAMOPY does not include the solver.

The two objects a user interacts with are AlamoTrainer, which trains a surrogate from data, and AlamoSurrogate, which wraps the resulting model so that it can be embedded in a Pyomo [12] flowsheet. A minimal session has the following shape:

from idaes.core.surrogate.alamopy import AlamoTrainer, AlamoSurrogate

trainer = AlamoTrainer(input_labels=[’x1’, ’x2’],
                       output_labels=[’z1’],
                       training_dataframe=data_training)
trainer.config.linfcns = 1
trainer.config.monomialpower = [2, 3]
trainer.config.modeler = 1
success, alm_surr, msg = trainer.train_surrogate()

Options are set as attributes of trainer.config and carry the same names and meanings as the corresponding ALAMO keywords documented in Section 6; $0\mathrm{-}1$ keywords are given as 0/1 or as True/False, and vector keywords are given as Python lists. Custom basis functions are supplied through trainer.config.custom_basis_functions, which is the ALAMOPY equivalent of the BEGIN_CUSTOMBAS section. The path to the ALAMO executable, the working directory, and the name of the generated .alm file can all be set through trainer.config as well, which is useful when the executable is not on the user’s path. A trained surrogate can be serialized to JSON and reloaded in a different script or notebook.

Because ALAMOPY is maintained by the IDAES project rather than by The Optimization Firm, its option names and calling conventions have changed between IDAES releases and may change again. Users should consult the current IDAES documentation [30] for the authoritative list of supported options and for the mapping between ALAMOPY results and ALAMO output. A Python interface built directly on ALAMO’s callable library (Section 4.3), and therefore free of the overhead of writing and parsing files, is planned for a future release.

The following parameters must be specified in the input file before any vector parameters described below.

The following parameters must be specified in the input file only after the scalar required parameters have already been specified.

XMIN and XMAX do not necessarily correspond to the minimum and maximum values of the data provided by the user to ALAMO. Instead, they should reflect the physically meaningful lower and upper bounds for the input variables. These values are used by ALAMO for scaling purposes as well as for generating samples when no preexisting data set is provided or, in general, when adaptive sampling is done.

ALAMO has many options, but a first model rarely requires more than a handful of them. The options that most often determine whether a run succeeds are the following.

• •

  The basis function flags (LINFCNS, MONOMIALPOWER, MULTI2POWER, LOGFCNS, EXPFCNS, and the rest of Section 6.6). ALAMO can only select from the basis functions it is given. The default set is a constant and linear terms, which is almost never enough.

• •

  MODELER (Section 6.7), which decides how model error is traded against model size.

• •

  SOLVEMIP (Section 6.7). With the default value of 0, ALAMO relies on enumeration and heuristics rather than on the embedded optimizer. Set it to 1 to obtain the best subset for the chosen fitness metric on problems that are too large to enumerate; expect higher CPU time.

• •

  SIMULATOR and MAXITER (Section 6.8), which together activate adaptive sampling. With the default MAXITER of 1, ALAMO builds one model from the data it is given and stops.

• •

  ZMIN, ZMAX, and the constrained regression options of Section 6.12, if the response is known to obey physical limits.

• •

  MAXTIME (Section 6.9), whose default of 1000 seconds is often the binding constraint on larger problems.

The remaining options are grouped by function in the subsections that follow. Any option not specified takes the default value listed in the tables. The parser is not case sensitive. For example, output variable labels Z1 and z1 are equivalent. For vector inputs, any items provided in excess of those required will be ignored. For example, if more than NINPUTS XLABELS are provided, the extra labels are ignored.

The options in this section describe the data available to ALAMO: preexisting measurements, points to be sampled before model building begins, points at which predictions are requested, and points reserved for testing.

If the parameter NDATA is set, then a data section must follow subsequently in the input file with precisely NDATA rows, one for each data point (pair of $x$ and $z$ values) specified in the following form:

BEGIN_DATA



END_DATA

If the parameter NPREDATA is set, then a data section must follow subsequently in the input file with precisely NPREDATA rows, each containing precisely NINPUTS values, thus corresponding to a point in the $x$-space, specified in the following form:

BEGIN_XPREDATA



END_XPREDATA

If the parameter NTESTDATA is set, a similar data section must be provided using a similar construct:

BEGIN_TESTDATA



END_TESTDATA

The options in this section control how input and output variables are named, scaled, and treated during model building.

Scaling matters more than it may appear. ALAMO’s fitness metrics and its mixed-integer subproblems are formulated in the units of the data, so input variables that differ by many orders of magnitude can distort basis function selection and degrade the numerics of the subproblems. Setting XSCALING to 1 is a safe default when the input ranges are dissimilar; XFACTOR gives explicit control when the ranges alone are not the right scaling.

If the parameter NTRANS is set to a positive entry, NTRANS functions must be provided using the construct:

BEGIN_TRANSFORMS



END_TRANSFORMS

where transformation functions are provided one per line. Line $k$ of a TRANSFORMS section provides an algebraic transformation that determines transformed output $k$ which, in turn, corresponds to output variable NOUTPUTS-NTRANS+$k$. The parser is not case sensitive and allows for Fortran functional expression in terms of the XLABELS and ZLABELS. For the types of functions supported, see the discussion under BEGIN_CUSTOMBAS above.

ALAMO builds its model as a linear combination of a subset of a user-specified collection of basis functions, as described in Section 3.3. The options in this section define that collection. A collection that is too small cannot represent the system; a collection that is far larger than the number of available measurements makes the subset selection problem harder and increases the risk of overfitting. A common starting point is linear terms plus a few monomial powers, enriched with pairwise products or logarithms only if the resulting model is not accurate enough.

The four limit options above cap families whose size grows quickly with the number of input variables. They are applied after the family has been generated, so they bound the size of the candidate set rather than the combinations considered when forming it.

If the parameters MONO, MULTI2, MULTI3, or RATIOS are set, the corresponding powers must also be specified as row vectors of corresponding length in the following way:

The entries of the above vectors must be space separated. As stated above, the user is not obligated to specify the parameters MONO, MULTI2, MULTI3, or RATIOS. If any of the corresponding power options are provided, ALAMO will count them and infer the total number of powers specified by the user.

If the parameter NCUSTOMBAS is set, user-specified basis functions must be provided using the construct:

BEGIN_CUSTOMBAS



END_CUSTOMBAS

where basis functions are provided one per line. The parser is not case sensitive and allows for Fortran functional expression in terms of the XLABELS. The following functions are currently accepted by the parser: addition, subtraction, multiplication, division, power ($\ast \ast$ and ^), abs, exp, log, log10, sqrt, sinh, cosh, tanh, sin, cos, tan, asin, acos, and atan. Other functions may be expressed in terms of the preceding operators and functions, e.g., $\min (a,b)=(a+b)/2-|a-b|/2$.

The options in this section control how ALAMO chooses a subset of the available basis functions, as described in Section 3.4.

A few remarks on the choices in this table.

Choosing a fitness metric. MODELER determines the trade-off between accuracy and model size; the metrics are defined in Section 3.4.1. BIC (the default) and RIC penalize model size aggressively and are the appropriate choices when the intent is to recover a compact, interpretable expression from limited data; BIC is the more conservative of the two when the number of candidate basis functions is comparable to the number of measurements. AICc and Mallow’s Cp penalize size less heavily and tend to return larger models that fit the training data more closely. HQC sits between BIC and AICc. MSE applies no size penalty at all and will select as many terms as the data permit; it is useful mainly as a reference point. SSEp and MADp with a nonzero CONVPEN allow the penalty to be set directly, and MADp is the right metric when the requirement is a uniform error bound rather than a small average error. When in doubt, build with two or three metrics and compare the resulting expressions on a test set. Note also that BIC, Mallow’s Cp, RIC, and ${\mathrm{SSE}}_p$ give rise to subproblems that ALAMO optimizes directly, whereas ${\mathrm{AIC}}_c$, HQC, and MSE require the model size to be parameterized and a separate subproblem to be solved for each size; on the benchmarks of Section 3.7 this made BIC more than two orders of magnitude faster than the latter group.

Screening. SCREENER reduces the number of candidate basis functions before the fitness metric is optimized. Screening is worth activating when the candidate set is large relative to the number of measurements, since the cost of the subset selection problem grows quickly with the number of candidates. Screening is a heuristic: it can discard a basis function that would have appeared in the optimal subset, so it should not be used when the candidate set is small enough to handle directly. The two methods are described in Section 3.4.4.

Heuristics and the optimizer. BUILDER and BACKSTEPPER are greedy heuristics that add and remove one basis function at a time. They are inexpensive and provide a starting point for the integer program. SOLVEMIP determines whether that integer program is actually solved. With SOLVEMIP set to 0, ALAMO returns the best model it can find by enumeration and heuristics, which is optimal when the candidate set is small enough to enumerate completely but not otherwise. Setting SOLVEMIP to 1 invokes the embedded BARON solver and is what makes the best subset selection guarantee described in Section 3.4.2 available in practice.

Limiting model size directly. DATALIMITTERMS and MAXTERMS impose hard caps on the number of terms. These are blunter instruments than the fitness metric but are useful when the surrogate must satisfy an external size requirement, for example because it will be embedded in a larger optimization model.

The options in this section govern the third step of the algorithm of Section 3.5, in which ALAMO decides where to collect additional measurements. Adaptive sampling is inactive unless MAXITER differs from 1 and either a SIMULATOR or a data bank (NBANK) is available.

The interaction between these options is worth stating explicitly. ALAMO first obtains NSAMPLE initial points using the design specified by INITIALIZER. It then builds a model, and, if MAXITER permits, calls the technique specified by SAMPLER to propose MAXPOINTS new points where the current model is suspected to be inaccurate. These points are drawn from the data bank if one exists and are otherwise requested from the SIMULATOR. If fewer than MINPOINTS of them can be evaluated, convergence is not assessed and the iteration is repeated. The cycle terminates when the model passes the tolerance tests of Section 6.9, when MAXITER iterations have been performed, or when MAXTIME is exhausted.

MAXTIME bounds a run in seconds, and seconds are not a property of the problem. The same input file stopped by the same MAXTIME on two machines, or on the same machine under different load, will in general stop at different places in the search and return different models. That is a nuisance when a run must be reproduced, and it is unacceptable when results are to be compared across machines or regression-tested.

ALAMO therefore also meters itself in work units, a machine independent measure of how much work has been done. One work unit is ${10}^5$ OLR-datapoints, and the counter is advanced by the three operations that dominate the cost of model building:

• •

  an ordinary linear regression over $m$ rows costs $m$ OLR-datapoints;

• •

  a constrained linear regression over $m$ rows of data and $m_c$ rows of discretized constraints costs $m+m_c$;

• •

  one call to the integer programming solver costs a fixed number of OLR-datapoints, so that the metering does not depend on how long the solver happens to run.

The count depends only on the sequence of operations ALAMO performs, not on how fast the machine performs them. Setting MAXWORK to a nonnegative value therefore makes termination deterministic: the run stops after the same amount of work, at the same point in the search, with the same incumbent model, on any machine and under any load. This is the reproducible counterpart of MAXTIME, and the two may be used together, in which case whichever limit is reached first ends the run.

Every run reports the work units it consumed, whether or not a budget was set, on the line

 Work units consumed   0.66E-03

of the termination summary described in Section 7. A convenient way to choose a budget is to run the problem once without one, read the figure off that line, and set MAXWORK to a fraction or a multiple of it. Fractional budgets are accepted, which matters because the individual figures are often small: the example above consumed less than a thousandth of a work unit.

A run that ends because the budget is exhausted terminates with condition 154, “Work-unit budget (MAXWORK) exhausted”, and reports the best model found up to that point; it is not an error.

In deciding whether to deactivate printing to the screen or file, users should consider that model coefficients are printed with two significant digits to the screen and with 23 digits to the listing file.

The contents of the listing file and of the trace file are described in Sections 7.2 and 7.3, respectively.

This section describes ALAMO’s constrained regression capabilities [6]; the underlying algorithm is described in Section 3.6. There are two distinct constrained regression capabilities implemented currently in ALAMO:

• •

  The ability to enforce constraints, such as bounds, on the response function.

• •

  The ability to enforce combinatorial constraints on the types of basis functions utilized, including constraints on groups of basis functions.

The primary options that control application of constrained regression to ALAMO’s response function are:

Custom constrained regression, i.e., constrained regression for enforcing conditions other than simple bounds, can be done by setting the option CRNCUSTOM:

If CRNCUSTOM is specified, the custom constraints themselves are described through a related section:

BEGIN_CUSTOMCON



END_CUSTOMCON

where, in each of CRNCUSTOM lines of this section, one would need to specify the output variable index $j$ associated with a custom constraint, followed by white space, followed by a function $g(x,z)$ expressed in terms of a Fortran expression of input and output variable labels. ALAMO will then enforce the constraint $g\le 0$ when building a model for output variable $j$. This feature supports nonlinear expressions in $g$ that include multiplication, division, powers, and exponentials.

The following are algorithmic options that control implementation aspects of the above constrained regression features. These options may be optionally set as follows:

In addition to imposing constraints on the response surface it produces, ALAMO has the ability to enforce constraints on groups of selected basis functions. This can be accomplished through ALAMO’s NGROUPS option:

If a positive NGROUPS is specified, the groups themselves must be specified through a related section:

BEGIN_GROUPS



END_GROUPS

where, in each line of this section, one would need to specify information of the form

Group-id Member-type Member-indices <Powers>

In this construct, each group is uniquely associated with a Group-id ranging from 1 to NGROUPS. Each line must contain three required parameters (Group-id, Member-type, Member-indices); the fourth parameter (Powers) is required only in the context of basis functions that involve powers. The syntax of this section must obey the following rules:

• •

  Each line pertains to a single group.

• •

  A group may be described over several lines, with each line restricted to describing a single type of component of the group.

• •

  Group-id is a nonnegative integer between 1 and NGROUPS that denotes the numerical id (index) of a group described (at least partly) in a line.

• •

  Member-type is an attribute for the member(s) of the group described in the specific line and can take any one of the values LIN, LOG, EXP, SIN, COS, MONO, MULTI2, MULTI3, RATIO, RBF, CUST, and CONST corresponding to different types of basis functions. In addition, Member-type may be set equal to GRP if it is desired to specify a group of groups.

• •

  Member-indices is used to specify the composition of a group in terms of indices of input variables and groups. The keyword CONST should not be followed by any Member-indices (as there is only one constant in the model). The keywords MONO, EXP, LOG, SIN and COS must be followed by exactly one index that has a value between 1 and NINPUTS corresponding to the input variable involved in the basis function; alternatively, a value of $-1$ may be used to denote that all input variables should be considered (with a specific power in the case of MONO). The keywords MULTI2, MULTI3 and RATIO must be followed by two, three, and two indices, respectively, indicating the input variables involved in the corresponding basis function; alternatively, an index of $-1$ may be used to denote that all possible input variable combinations in any of these basis functions should be considered. The keyword LIN may be followed by as many as NINPUTS Member-indices and specifies which linear terms of the model are included in the group; alternatively, an index of $-1$ may be used to denote that all possible linear terms are included in the group. Similarly, the keywords CUST and RBF may be followed by as many as NCUSTOMBAS and NDATA Member-indices and an index of $-1$ may be used to denote all possible custom basis functions and RBFs in a group. As many as NGROUPS$-1$ indices may follow the keyword GRP in order to specify which groups form a group.

• •

  When Member-type is one of MONO, MULTI2, MULTI3, or RATIO, in addition to Member-indices, the input line must specify the power(s) involved in the group; if power equals $-1111$, all powers are considered.

• •

  Membership in a group is non-exclusive; a basis function or group may belong to multiple groups.

Once the number of groups has been specified and each group has been described through the GROUPS construct, group constraints can be specified through the GROUPCON section:

BEGIN_GROUPCON



END_GROUPCON

ALAMO permits different group constraints to be imposed on different output variables. Each line of the GROUPCON section is dedicated to a group-output variable combination and has the following information:

Group-id Output-id Constraint-type Integer-parameter

The rules for completing this section are as follows:

• •

  Each line describes a single group constraint.

• •

  Group-id is a nonnegative integer between 1 and NGROUPS that denotes the numerical id (index) of the primary (and sometimes only) group involved in the constraint.

• •

  Output-id is a nonnegative integer between 1 and NOUTPUTS for which the constraint will be imposed; a value of $-1$ can be used in this entry to denote that the constraint should be enforced for all output variables.

• •

  Constraint-type is a string descriptor that can take any one of the following values:

  • –

    NMT: to denote a no-more-than constraint, i.e., require that no more than Integer-parameter members of this group should be selected in the model.

  • –

    ATL: to denote an at-least constraint, i.e., require that at least Integer-parameter members of this group should be selected in the model.

  • –

    REQ: to require that, if the primary group is selected, then the group with id equal to Integer-parameter should also be selected.

  • –

    XCL: to require that, if the primary group is selected, then the group with id equal to Integer-parameter should not be selected.

• •

  A group may appear in more than one constraint. This flexibility coupled with the fact that input variables and groups may belong to multiple groups allows us to enforce sparsity constraints on model attributes within groups, between groups, and groups organized in clusters, trees, or any other structure.

The screen output below is obtained for problem e1.alm.

 ***************************************************************************
 ALAMO version 2026.9.10. Built: WIN-64 2026-09-10 18:09:46

 ALAMO is a product of The Optimization Firm. For more information and
 related optimization and data analytics products visit https://minlp.com.

 If you use this software, please cite:
 Cozad, A., N. V. Sahinidis and D. C. Miller,
 Automatic Learning of Algebraic Models for Optimization,
 AIChE Journal, 60, 2211-2227, 2014.

 ALAMO is powered by the BARON software from http://www.minlp.com/.
 ***************************************************************************
 Reading input data
 Licensee: Nick Sahinidis at The Optimization Firm, LLC, niksah@minlp.com.
 ***************************************************************************
 Checking input consistency and initializing data structures
 Warning: eliminating basis log(X1)

 Step 0: Initializing data set
 User provided an initial data set of 11 data points
 We will sample no more data points at this stage
 ***************************************************************************
 Iteration 1 (Approx. elapsed time 0.31E-01 s)

 Step 1: Model building using BIC

 Model building for variable Z
 ----
 BIC = -0.100E+31 with Z = X1^2

 Calculating quality metrics

 Quality metrics for output Z
 ----------------------------
 SSE OLR:           0.00
 SSE:               0.00
 RMSE:              0.00
 R2:                1.00
 R2 adjusted:       1.00
 Model size:        1
 BIC:               -0.100E+31
 Cp:                -9.00
 AICc:              -0.100E+31
 HQC:               -0.100E+31
 MSE:               0.00
 SSEp:              0.00
 RIC:               3.89
 MAD:               0.00

 Work units consumed   0.66E-03
 Total execution time 0.31E-01 s
 Times breakdown
     OLR time:        0.0 s in 6 ordinary linear regression problems
     MINLP time:      0.0 s in 0 optimization problems
     Simulation time: 0.0 s to simulate 0 points
     All other time:  0.31E-01 s in 1 iterations

 Normal termination
 ***************************************************************************

The software first reports the version, platform, and compilation date of the executable, followed by credits. Then, after reading the input data, a consistency check is run on the problem data and, if passed, the data structures are initialized. In this specific example, a warning is issued that logarithmic basis functions are not considered since the input variable is allowed to take negative values. Subsequently, information is provided for all algorithmic steps. During initialization (Step 0), it is reported that 11 data points are used for sampling and that no sampling is done in addition to using the preexisting data set. In Step 1, the model is built in stages. Earlier in the search, the perfect model $z=x^2$ is identified. Since there is no simulator provided and NBANK is zero, there is no adaptive sampling and execution terminates here after reporting the ALAMO work units consumed and a detailed breakdown of CPU times for the different algorithmic steps, including the number of calls to the optimizer (0 in this example) and the simulator (0 in this example). There are no calls to an optimizer in this example because the problem is small enough to be solved faster by complete enumeration.

Unless PRINT_TO_FILE is set to 0, ALAMO writes a listing file whose name is that of the input file with the extension replaced by ‘.lst.’ The listing file is always placed in the execute directory, even when the input file resides elsewhere.

The listing file contains the same sequence of sections as the screen output. The two differ in one respect that matters when the model is to be used elsewhere: model coefficients are printed with two significant digits to the screen and with 23 digits to the listing file. The screen output is therefore suitable for reading, and the listing file is the copy to use when transcribing a model into another tool. The listing file can also be read back by the GUI in order to reconstruct the problem inputs and results.

Setting TRACE to 1 directs ALAMO to append a compact, machine-readable summary of the run to the file named by TRACEFNAME. This is the output to use when ALAMO is run over many problems and the results are to be collected and compared automatically.

The file begins with a header line marked by a # that names the fields of each subsequent line. After the header, ALAMO writes one line per output variable per data set. Data sets are identified by the numbers 0 for the observed data set, $-1$ for a user-provided test data set, and $1,\mathrm{\dots },$ NTESTSETS for the test sets generated by ALAMO. Because summaries are appended rather than overwritten, a trace file accumulates the results of successive runs; delete or rename it between experiments if that is not the intent.

ALAMO ships as a shared library (libalamo.so on Linux, libalamo.dylib on macOS, libalamo.dll on Windows) that can be linked into a calling program. Everything the input file can express can be expressed through this interface, and nothing needs to be written to disk: the caller sets the options and the data in memory, asks for a solve, and reads the model back as a string.

Every entry point exists in two flavors:

• •

  a C-callable routine alm_xxx, declared in the header file alminterface.h reproduced in Appendix A; and

• •

  a native Fortran routine alm_xxx_f, taking ordinary Fortran character, integer and real arguments, which Fortran callers may call directly with no iso_c_binding declarations.

The C routine is a thin shim: it marshals C types to Fortran types and calls the _f routine of the same base name. The two are therefore always in step, and a Fortran program may use whichever it prefers. ALAMO holds one problem at a time; alm_init resets all of it, so a program that builds several models in succession calls alm_init before each. The library is not reentrant and a single problem must not be driven from more than one thread at a time.

Every function that returns an int returns 0 on success and an ALAMO status code otherwise. The codes are those of Section 10; codes 142 through 159 are raised only by this interface, and alm_get_status_message turns any of them into the text printed there.

The program below builds the Quickstart problem of Section 2, the same eleven points and the same basis set, without reading e1.alm.

/* e1api.c -- the Quickstart problem e1.alm, built through the ALAMO
   callable library instead of being read from a file.                  */

#include <stdio.h>
#include <stdlib.h>
#include "alminterface.h"

int main(void)
{
    /* the 11 data points, one row per point: x followed by z */
    static const double data[22] = {
        -5.0, 25.0,   -4.0, 16.0,   -3.0,  9.0,   -2.0,  4.0,
        -1.0,  1.0,    0.0,  0.0,    1.0,  1.0,    2.0,  4.0,
         3.0,  9.0,    4.0, 16.0,    5.0, 25.0
    };
    static const double xmin[1]  = { -5.0 };
    static const double xmax[1]  = {  5.0 };
    static const double mono[2]  = {  2.0, 3.0 };

    int status, len;
    double wu;
    char msg[256], *model;

    alm_init();

    if ((status = alm_set_int("ninputs",  1)))              goto failed;
    if ((status = alm_set_int("noutputs", 1)))              goto failed;
    if ((status = alm_set_real_vector("xmin", xmin, 1)))    goto failed;
    if ((status = alm_set_real_vector("xmax", xmax, 1)))    goto failed;

    if ((status = alm_set_int("ndata", 11)))                goto failed;
    if ((status = alm_set_real_vector("data", data, 22)))   goto failed;

    if ((status = alm_set_int("linfcns", 1)))               goto failed;
    if ((status = alm_set_int("logfcns", 1)))               goto failed;
    if ((status = alm_set_int("expfcns", 1)))               goto failed;
    if ((status = alm_set_int("sinfcns", 1)))               goto failed;
    if ((status = alm_set_int("cosfcns", 1)))               goto failed;
    if ((status = alm_set_int("mono", 2)))                  goto failed;
    if ((status = alm_set_real_vector("monomialpower", mono, 2)))
                                                            goto failed;

    if ((status = alm_alamo()) != 0) goto failed;

    if (alm_get_model_length(1, &len) == 0) {
        model = malloc((size_t) len + 1);
        if (model && alm_get_model(model, 1) == 0)
            printf("model: %s\n", model);
        free(model);
    }
    if (alm_get_real("wuused", &wu) == 0)
        printf("work units consumed: %g\n", wu);
    return 0;

 failed:
    alm_get_status_message(status, msg, (int) sizeof msg);
    printf("ALAMO failed with status %d: %s\n", status, msg);
    return 1;
}

Besides the usual ALAMO output, this program prints

model: X1^2
work units consumed: 0.00066

recovering $z=x^2$ exactly, and consuming the same 0.66E-03 work units that the run of Section 7 reports. It must, the two runs being the same computation.

Fortran callers can use the _f entry points directly. No module and no include file is needed: the routines take ordinary Fortran types, and the program below declares them as externals.

      program e1api

      use iso_fortran_env, only: real64, int32
      implicit none

      integer(int32) :: alm_set_int_f, alm_set_real_vector_f
      integer(int32) :: alm_alamo_f, alm_get_model_length_f
      integer(int32) :: alm_get_model_f, alm_get_real_f
      external :: alm_set_int_f, alm_set_real_vector_f
      external :: alm_alamo_f, alm_get_model_length_f
      external :: alm_get_model_f, alm_get_real_f
      external :: alm_dummy_simulator

      real(real64) :: data(22), xmin(1), xmax(1), mono(2), wu
      integer(int32) :: st, ml
      character(len=:), allocatable :: model
      character*256 :: msg

*     the 11 data points, one row per point: x followed by z
      data data / -5.d0, 25.d0,  -4.d0, 16.d0,  -3.d0,  9.d0,
     $            -2.d0,  4.d0,  -1.d0,  1.d0,   0.d0,  0.d0,
     $             1.d0,  1.d0,   2.d0,  4.d0,   3.d0,  9.d0,
     $             4.d0, 16.d0,   5.d0, 25.d0 /
      data xmin / -5.d0 /
      data xmax /  5.d0 /
      data mono /  2.d0, 3.d0 /

      call alm_init_f()

      st = alm_set_int_f(’ninputs’, 1)
      if (st /= 0) goto 900
      st = alm_set_int_f(’noutputs’, 1)
      if (st /= 0) goto 900
      st = alm_set_real_vector_f(’xmin’, xmin, 1)
      if (st /= 0) goto 900
      st = alm_set_real_vector_f(’xmax’, xmax, 1)
      if (st /= 0) goto 900

      st = alm_set_int_f(’ndata’, 11)
      if (st /= 0) goto 900
      st = alm_set_real_vector_f(’data’, data, 22)
      if (st /= 0) goto 900

      st = alm_set_int_f(’linfcns’, 1)
      if (st /= 0) goto 900
      st = alm_set_int_f(’logfcns’, 1)
      if (st /= 0) goto 900
      st = alm_set_int_f(’expfcns’, 1)
      if (st /= 0) goto 900
      st = alm_set_int_f(’sinfcns’, 1)
      if (st /= 0) goto 900
      st = alm_set_int_f(’cosfcns’, 1)
      if (st /= 0) goto 900
      st = alm_set_int_f(’mono’, 2)
      if (st /= 0) goto 900
      st = alm_set_real_vector_f(’monomialpower’, mono, 2)
      if (st /= 0) goto 900

      st = alm_alamo_f(alm_dummy_simulator)
      if (st /= 0) goto 900

      if (alm_get_model_length_f(1, ml) == 0) then
         allocate(character(len=ml) :: model)
         if (alm_get_model_f(model, 1) == 0) then
            print *, ’model: ’ // trim(model)
         endif
      endif
      if (alm_get_real_f(’wuused’, wu) == 0) then
         print *, ’work units consumed: ’, wu
      endif
      stop

 900  call alm_get_status_message_f(st, msg)
      print *, ’ALAMO failed with status ’, st, ’: ’ // trim(msg)

      end program e1api

The native alm_alamo_f takes one argument, the simulator procedure. A program that has registered a callback with alm_set_simulator_f, or that needs no simulator at all as here, passes alm_dummy_simulator, a do-nothing procedure the library exports for the purpose.

A Fortran 90 program may also call the C entry points, which is what a program does when it wants to exercise exactly the ABI a C caller sees. The interfaces do not have to be written out: the file alminterface.inc, shipped in the include directory and reproduced in Appendix B, declares all of them. Include it in the declaration part of the program, after use iso_c_binding, whose kind parameters it refers to.

!-----------------------------------------------------------------------
!  e1api.f90 -- the Quickstart problem e1.alm, built through the ALAMO
!  callable library using the C-bound entry points.  The interfaces come
!  from alminterface.inc, which is shipped in the include directory, so
!  nothing has to be declared by hand.
!-----------------------------------------------------------------------
program e1api
  use iso_c_binding
  implicit none

  include ’alminterface.inc’

  ! the 11 data points, one row per point: x followed by z
  real(c_double), parameter :: dat(22) = [ &
       -5.0d0, 25.0d0,  -4.0d0, 16.0d0,  -3.0d0,  9.0d0, &
       -2.0d0,  4.0d0,  -1.0d0,  1.0d0,   0.0d0,  0.0d0, &
        1.0d0,  1.0d0,   2.0d0,  4.0d0,   3.0d0,  9.0d0, &
        4.0d0, 16.0d0,   5.0d0, 25.0d0 ]
  real(c_double), parameter :: xmin(1) = [ -5.0d0 ]
  real(c_double), parameter :: xmax(1) = [  5.0d0 ]
  real(c_double), parameter :: mono(2) = [  2.0d0, 3.0d0 ]

  character(len=256) :: buf
  real(c_double) :: wu
  integer(c_int) :: st, ml
  integer :: i

  call alm_init()

  st = alm_set_int(cstr(’ninputs’), 1);            if (st /= 0) goto 900
  st = alm_set_int(cstr(’noutputs’), 1);           if (st /= 0) goto 900
  st = alm_set_real_vector(cstr(’xmin’), xmin, 1); if (st /= 0) goto 900
  st = alm_set_real_vector(cstr(’xmax’), xmax, 1); if (st /= 0) goto 900

  st = alm_set_int(cstr(’ndata’), 11);             if (st /= 0) goto 900
  st = alm_set_real_vector(cstr(’data’), dat, 22); if (st /= 0) goto 900

  st = alm_set_int(cstr(’linfcns’), 1);            if (st /= 0) goto 900
  st = alm_set_int(cstr(’logfcns’), 1);            if (st /= 0) goto 900
  st = alm_set_int(cstr(’expfcns’), 1);            if (st /= 0) goto 900
  st = alm_set_int(cstr(’sinfcns’), 1);            if (st /= 0) goto 900
  st = alm_set_int(cstr(’cosfcns’), 1);            if (st /= 0) goto 900
  st = alm_set_int(cstr(’mono’), 2);               if (st /= 0) goto 900
  st = alm_set_real_vector(cstr(’monomialpower’), mono, 2)
  if (st /= 0) goto 900

  st = alm_alamo();                                if (st /= 0) goto 900

  if (alm_get_model_length(1, ml) == 0 .and. ml < len(buf)) then
     buf = ’’
     if (alm_get_model(buf, 1) == 0) then
        i = index(buf, char(0))
        if (i > 0) buf(i:) = ’’
        print *, ’model: ’ // trim(buf)
     endif
  endif
  if (alm_get_real(cstr(’wuused’), wu) == 0) then
     print *, ’work units consumed: ’, wu
  endif
  stop

900 print *, ’ALAMO failed with status ’, st

contains

  ! append the NUL that the C entry points expect
  function cstr(s) result(c)
    character(len=*), intent(in) :: s
    character(len=len(s)+1, kind=c_char) :: c
    c = s // char(0)
  end function cstr

end program e1api

Two points of technique are worth noting. The C entry points expect NUL-terminated strings, which the small cstr helper supplies, and the model comes back NUL-terminated as well, so the trailing NUL is located and removed before the string is printed.

Fortran callers who would rather work in native Fortran types need none of this. The alm_xxx_f entry points of Section 8.4 take ordinary Fortran arguments and may be declared as plain externals, with no include file and no iso_c_binding.

Three ingredients are needed: the driver, the shared library, and, for C and for Fortran 90 drivers that include the interfaces, the file from the include directory. The commands below assume ALAMO is installed under /usr/local/alamo and that the driver is one of e1api.c, e1api.f or e1api.f90 of the preceding sections. Adjust the installation path and nothing else.

A model with many terms and an excellent training fit is a symptom of a fitness metric that is too permissive, of a candidate basis set that is too rich for the amount of data available, or of both. Move MODELER toward a criterion that penalizes size more heavily, such as BIC or RIC; reduce the basis set; impose an explicit cap through MAXTERMS; or supply a test set through NTESTDATA or NTESTSAMPLE so that the discrepancy between training and test error becomes visible. Note also that with SOLVEMIP set to 0 the model returned is the output of enumeration and greedy heuristics, which on a large candidate set may be neither small nor accurate.

The complementary failure is a model that is small and inaccurate. In many cases, the cause is a candidate basis set that cannot represent the system. ALAMO selects only from the functions it is offered, and the default set consists of a constant and linear terms. Add monomial powers, pairwise or three-way products, or logarithmic and exponential terms as the physics suggests, and consider supplying problem-specific functions through the BEGIN_CUSTOMBAS section. If the basis set is adequate and the error remains large, the training data are likely too sparse; activate adaptive sampling as described below.

Adaptive sampling stops when the tolerances of Section 6.9 are met, when MAXITER iterations have elapsed, or when MAXTIME is exhausted. A run that ends on time or on iteration count rather than on tolerance usually means that TOLMAXERROR is tighter than the model class can achieve, or that the simulator is returning values that the model cannot reconcile, for example because the response is discontinuous or noisy. Loosen TOLMAXERROR, enrich the basis set, or accept the model produced at the iteration limit after inspecting its test error.

Termination code 123 indicates that the simulator could not be found; check that the executable named by SIMULATOR is on the path or that the full path was supplied, and recall that ALAMO runs the simulator in a scratch directory it creates, so the simulator must not depend on relative paths. Code 124 indicates that the simulator failed MAXSIM times in succession. Codes 37 and 38 point at the SIMIN and SIMOUT files themselves. Code 37 means that ALAMO was unable to write on disk, perhaps due to a full disk or permission issues. Code 38 means that the simulator did not write the expected number of values, because of similar I/O issues or bugs in the simulator itself. Codes 42 and 43 indicate that the simulator returned NaN; for an output variable that cannot be computed at a given point, the simulator must return PRESET rather than NaN.

With NUMLIMITBASIS set to 1, ALAMO discards basis functions that are not numerically well defined over the box between XMIN and XMAX. Logarithmic and, on some data, ratio basis functions are the usual casualties, and a warning to that effect appears in the screen and listing output. If those functions are needed, restrict XMIN and XMAX to a region on which they are defined rather than turning NUMLIMITBASIS off. Note that XMIN and XMAX describe the physically meaningful range of each input and are not required to coincide with the range of the supplied data.

Constrained regression enforces ZMIN, ZMAX, and any custom constraints at a growing finite set of points, adding CRNVIOL violated points per constraint per iteration for at most CRMAXITER iterations. A run that exits with a model still violating a bound usually needs a larger CRMAXITER or CRNVIOL, or a basis set rich enough to satisfy the constraint at all. Note that satisfaction is enforced only to within CRTOL, and that constrained regression is not supported together with trigonometric basis functions (code 134) and requires an optimizer (code 73).

Termination code 132 means that the current combination of options requires the embedded mixed-integer solver but that it is not available. This is most commonly encountered with SOLVEMIP set to 1 or with constrained regression active. Verify that the BARON executable that came with the ALAMO distribution is in the same location as your ALAMO executable or callable library. In general, ensure that all executables and libraries that come with your ALAMO install remain together in the same location on your computer.