whenever you notice it.''
(1 Thess. 5:21-22)
This file is part of my global optimization web site. It contains sections on
A recent Handbook of Test Problems in Local and Global Optimization by C. Floudas et al. contains a large collection of test problems for local and global optimization problems of types including
Interesting Properties of Moré/Garbow/Hillstrom Test Functions (multiple minima, nearly flat plateaus, regions where roundoff matters, etc.)
Fortran Code for Moré/Garbow/Hillstrom test problems and gradients, and for some others
Minimum Function Values and Gay's Bounds for Moré/Garbow/Hillstrom test problems
For large scale problems in local unconstrained optimization, the
standard is the collection of
MINPACK-2 Test Problems for Large-Scale Optimization; the actual code for the problems is in tprobs*
Data Fitting Test Problems (by Klaus Schittkowski)
Cambridge Cluster Database (The Lennard-Jones cluster problem and more realistic cluster problems)
Test Functions and Benchmarks for Genetic Algorithms
(collected by Leo Lazauskas)
Of course, other global optimization algorithms should be able to solve these, too, and also be able to compete on the global optimization test set developed for the First International Contest on Evolutionary Optimization. It contains five problems, each in a 5- and 10-dimensional version.
Test Problems from the Second International Contest on Evolutionary Optimization (includes also one constrained example, with nonempty interior)
Since the above links are no longer working, here is code for ICEO1 and ICEO2 tests: iceo.tar.gz(C and Matlab, 16K) iceo.zip(C code only, 12K) and the global optima for ICEO1: iceo.txt
Simple Challenges for Global Optimization Codes exhibiting features missing from many of the global optimization test problems in vogue
PGAPack set of test problems in Fortran and C,
More Pictures (ps, 147K)
Test Functions for Evolutionary Algorithms (William M. Spears)
The Moré/Garbow/Hillstrom test problems are standard test problems for the case of continuous variables when only one solution is requested. For testing the case where all solutions are wanted, this test set is probably too easy.
Test Database of Polynomial Systems (by Jan Verschelde)
Polynomial test suite (by D. Bini and B. Mourrain)
The COPRIN examples page for systems of equations and inequalities
The following links generally address the case of discrete variables.
Constraint Satisfaction Benchmarks, a list of links
14 Binary Constraint Satisfaction Programs in C with four test problem classes (graph coloring, n-queens, totally random, random with a solution)
TESTNONLIN - Nonlinear Equation Tests in Fortran 90
The test problems are part of the GLOBAL Library of global optimization test problems coded as GAMS models.
The traditional test set for constrained local optimization is that by Hock and Schittkowski. It was designed for testing (low-dimensional) local optimization algorithms, but many of these test problems are nonconvex and possess several local minima with different objective function values. There is a Fortran 77 implementation of the collection.
A huge collection of (often large-scale) test problems is
CUTEr suite of Fortran subroutines, scripts and test problems
for linear and nonlinear optimization
CUTEr Matlab interface
another CUTEr Matlab interface
It is advisable to select test problems with specific characteristics using Rolf Felkel's CUTE Problem Selector
The Hock/Schittkowski problems and part of the CUTE problems
are also part of the collection of
Nonlinear Optimization Models in AMPL (collected by Bob Vanderbei)
with many additional problems from practical applications.
COPS: Large-Scale Nonlinearly Constrained Optimization Problems
(in AMPL and C)
Another, more recent test set is the COPS test results
The GAMS Model Library Index
nonlinear programming problems from practical applications, coded in GAMS
COMPleib, Constraint Matrix-Optimization Problem Library
a collection of test examples for nonlinear semidefinite programs, control system design and related problems
Nonlinear programming test function suite (for evolutionary computation)
QPlib2014, Quadratic Programming Library
A library of test problems for constrained optimization problems where the objective function and all constraints are linear or quadratic. (but no LPs)
Concave Minimization Test Problems (Jacobsen and Moshirvaziri)
``All of the problems have non-global local minima.''
A constrained problem difficult for genetic algorithms ((Quanshi Xia)
MCPLIB (nonlinear mixed complementarity problems)
A problem by Andy Keane with active constraints (with C code)
SDPLIB (semidefinite programming test problems)
OR-Library Nonlinear Programming Test Problems [currently empty, Jan 9 1997]
Some of the online papers by Baker Kearfott contain constrained global optimization problems together with test results for his branch and bound algorithm.
The nonlinear programming FAQ contains a list of papers with optimization test problems. The papers are not online, however.
There is also a book,
CSPLIB, a problem library for constraints
DIMACS Challenge on Cliques, Coloring and Satisfiability (hard combinatorial optimization problems)
OR-Library test problems for quadratic assignment, etc.
QAPLIB (Quadratic Assignment Problem Library)
Donald Knuth's Stanford GraphBase
Generating Traveling Salesman Problems with known solutions
Exact Ground States of Spin Glasses
LOLIB library of sample instances for the linear ordering problem
sac-94-suite of 0/1 integer programming problems
(by J. Heitkoetter)
(multiple knapsack problems)
A hard mixed integer problem (by D. Bienstock), with approx. 400 continuous variables and 56 (0,1)-variables
QBFlib, The Quantified Boolean Formulas Satisfiability Library
Planning & Scheduling Benchmarks: Resource Constrained Project Scheduling
planning and scheduling benchmark
Staff Rostering Benchmark Data Sets
Vehicle routing instances
VRPTW benchmark problems (routing and scheduling)
Planning & Scheduling Benchmarks (B.R. Fox and M. Ringer)
MINLP Library for mixed integer nonlinear programming problems
Test Problems for Mixed Integer Nonlinear Programming in AMPL or SIF
Stochastic Programming Problems (Robert Entriken)
Comparing Solution Methods for Dynamic Equilibrium Economies (S.B. Aruoba et al.)
Multidisciplinary Optimization Test Suite (Natalia Alexandrov)
Optimization Software and Test Problems (an extensive list of links by Tomomi Matsui)
I'd like to encourage a common format for reporting results and invite your ideas about what you would like to see in a result database. To get you started, look at my current ideal for Test Result Presentation.
You might also be interested in my suggestions for a global optimization contest and in Benchmarking Optimization Software with COPS (by Jorge Moré's group at Argonne)
Recent Papers and Preprints
my home page
Arnold Neumaier (Arnold.Neumaier@univie.ac.at)