Package {OSCARS}

OSCARS-II bound constrained global optimization

Description

Performs black-box optimization of a general function subject to bounds on the unknown parameters using a variant of the OSCARS-II algorithm (Price, Reale and Robertson (2020) <doi.org/10.1007/s10898-020-00928-6>). If all bounds are finite, Oscars acts as a global optimization algorithm. It has been adapted to handle infinite upper and lower bounds, in which case the method has the characteristics of a local method for nonsmooth problems. Oscars does not use or assume the existence of derivatives of the objective function. It is a low overhead method for cheaply evaluated black-box functions. Black-box optimization methods for arbitrary functions do not and cannot provide certificates of optimality if halted after a finite amount of time.

Oscars is a stochastic direct search method which uses only function values at selected points. It generates a finite sequence of nested boxes around a control point, and randomly samples each box once, in turn. A new set of nested boxes is formed if the current set is exhausted or a point better than the control point is found. In the latter case the better point replaces the control. Initially the control point is set to the better of an internal initial point and a user supplied start point (if given).

From time to time the control is reset alternately to a random point, or to the best known point. Each reset marks the end of one cycle and the start of the next. All even numbered cycles start with control points chosen randomly from the feasible region. All odd numbered cycles (other than the first) set the control point equal to the best known point.

Oscars either performs a fixed number of function evaluations, or it halts if progress stalls for a significant period of time. In both cases it returns the best known point and the function evaluated at that point.

Usage

oscars(
  fname,
  n,
  lwr,
  upr,
  ...,
  start = NULL,
  controls = oscars.control(),
  progress = TRUE
)

Arguments

Value

A list containing results of the optimization. This list consists of the following components:

• par: vector containing the best known parameters

• value: The minimized (or maximized) function value

• evaluations: The number of function evaluations used

• cycles: The number of cycles used

• convergence: 0 if the function and parameter tolerances have been reached; 1 if tolerances have not been reached but function evaluation budget has been exhausted; 2 if bounds are inconsistent.

• message: A text string explaining the value in convergence.

• controls: The values of the controls provided to oscars

Examples

# Camel function with global minima of f = -1.0316 at
# (0.0898,0.7127) and (0.0898,-0.7127) with four other local minima
camel <- function(par) {
  x = par[1]
  y = par[2]
  f = 4*x^2 - 2.1*x^4 + (1/3)*x^6 + x*y + 4*(y^4-y^2)
  return(f) }
out <- oscars(camel, n = 2, lwr = c(-5,-5), upr = c(5,5))

# How to use repeated upper and lower bounds.
# Bird function in 2 dimensions.  Global minimum = -106.7645367198
bird <- function(par)  {
  x1 = par[1];  x2 = par[2]
  f = sin(x1)*exp((1-cos(x2))^2) + cos(x2)*exp((1-sin(x1))^2) + (x1-x2)^2
  return(f)
} # end of bird function
out <- oscars(bird, 2, -10, 50)

# Hosaki function with global minimum of -2.3458 at (4,2) and one local minimum
hosaki <- function(par)  {
  x = par[1]
  y = par[2]
  f = (1 - 8*x + 7*x^2 - (7/3)*x^3 + (1/4)*x^4)*y*y*exp(-y)
  return(f) }
out <- oscars(hosaki, 2, 0, upr = c(5,6))

# The proper way to specify control parameters.
out <- oscars(hosaki, 2, lwr = c(0,0), upr = c(5,6),
  controls = oscars.control(nfmax = 100000, fTol=10*oscars.control()$fTol))

# An example of where the full function evaluation budget is used.
out <- oscars(hosaki,2,0,5,controls = oscars.control(nfmax=10000,fTol=-1))

# how to pass other values to the objective function
# Rosenbrocks "banana" function with global minimum of zero at (a, a^2)
rosenbrock <- function(par, a = 1, b = 100){
   f = (a - par[1])^2 + b*(par[2] - par[1]^2)^2
   return(f)
}
out <- oscars(rosenbrock, 2, -3, 3, a = 0.5)

# Providing a user start point to the algorithm.
# Weka_1 function with global minimum of 0 wherever x[1] = -1 and a local
# minimum at the origin.  Dimension n is arbitrary with bounds -1 <= x <= 2
weka_1 <- function(par)   {
  f1 = 1 + sqrt( sum( par^2 ))
  f2 = 4*par[1] + 4
  f = min(f1,f2)
  return(f)
}
out <- oscars(weka_1, 10, -1, 2, start = 0)

# An example of the use of infinite bounds.
# Active faces is a nonsmooth function with global minimum = 0 at the
# origin.   Standard bounds are 0 <= par <= 5.  Solution is on boundary.
activefaces <- function(par)  {
f1 = max( log( abs(par) + 1) )
f2 = log( abs( sum(par) ) + 1)
f = max(f1,f2)
return(f)
}
out <- oscars(activefaces, 10, 0, Inf, start = 4)

Control parameters for oscars routine

Description

Provides control over oscar parameters, such as number of iterations, tolerance, etc.

Usage

oscars.control(
  nfmax = 50000,
  infol = 1,
  DoMax = FALSE,
  fTol = 1e-06,
  xTol = 1e-08
)

Arguments

Details

A subset of parameters can be specified. All non-specified parameters revert to their defaults. No parameter abbreviations.

Value

A named list of control parameters for oscars.

Examples

oscars.control()  # default values
oscars.control(nfmax = 100000) # bump iteration budget
oscars.control(xTol = 10*oscars.control()$xTol) # increase xTol

OSCARS-II-quasi-Newton for bound constrained global optimization

Description

Performs global optimization of a general function subject to bounds on the unknown parameters using a variant of the algorithm in (Price, Reale and Robertson (2025) <doi.org/10.1007/s43069-024-00403-y>). The method is designed for functions which are continuously differentiable, but will run on black-box functions where only function values are available. It incorporates aspects of the direct search method OSCARS-II, guaranteeing eventual convergence even on black-box continuous functions. From time to time quasi-Newton steps are performed to accelerate convergence and improve the accuracy of the estimated optimizer. The algorithm will use analytic gradients if provided, otherwise it will estimate them via finite differences.

If all bounds are finite, oscarsQN acts as a global optimization algorithm. It has been adapted to handle infinite upper and lower bounds, in which case the method has the characteristics of a local method. Black-box methods for global optimization of arbitrary functions do not and cannot provide certificates of optimality if halted after a finite amount of time, even if the gradient is available at sample points.

OscarsQN is a stochastic direct search method which uses function values and gradients at selected points. It generates a finite sequence of nested boxes around a control point, and randomly samples each box once, in turn. Once the current set is exhausted or a point better than the control point is found the algorithm performs one quasi-Newton step and constructs a new set of nested boxes. If a better point than the control is found, it replaces the control. Initially the control point is set to the better of an internal initial point and a user supplied start point (if given).

From time to time the control is reset alternately to a random point, or to the best known point. Each reset marks the end of one cycle and the start of the next.

OscarsQN either performs a fixed number of function evaluations, or it halts if a user specified target value has been reached, or the same best locally optimal point has been seen a prescribed number of times. It returns the best known point and the function value at that point.

Usage

oscarsQN(
  fname,
  gname = NULL,
  n,
  lwr,
  upr,
  ...,
  start = NULL,
  progress = TRUE,
  controls = oscarsQN.control()
)

Arguments

Value

A list containing results of the optimization. This list consists of the following components:

• par: vector containing the best known parameters

• value: The minimized (or maximized) function value

• evaluations: The number of function evaluations used. Function evaluations used in calculating finite difference estimates of the gradient are included in this total, but any analytic gradient calculations are not.

• cycles: The number of cycles used.

• convergence: 0 if the function and parameter tolerances have been reached; 1 if tolerances have not been reached but function evaluation budget has been exhausted; 2 if bounds are inconsistent.

• message: A text string explaining the value in convergence.

• numberKKTpoints: Number of Karush-Kuhn-Tucker (KKT) points found. KKT points are potential minimizers.

• numberbestKKTpoints: Number of KKT points found which take the best known function value (within tolerance).

• controls: The values of the controls provided to oscarsQN.

Examples

# Camel function with global minima of f = -1.0316 at
# (0.0898,0.7127) and (0.0898,-0.7127) with four other local minima
camel <- function(par) {
  x = par[1]
  y = par[2]
  f = 4*x^2 - 2.1*x^4 + (1/3)*x^6 + x*y + 4*(y^4-y^2)
  return(f) }

camelgrad <- function(par) {
  x = par[1]
  y = par[2]
  g = c(0, 0)
  g[1] = 8*x - 8.4*x^3 + 2*x^5 + y
  g[2] = x + 16*y^3 - 8*y
  return(g) }
out <- oscarsQN(camel, camelgrad, n = 2, lwr = c(-5,-5), upr = c(5,5))


# Bird function in 2 dimensions.  Global minimum = -106.7645367198
bird <- function(par)  {
  x1 = par[1];  x2 = par[2]
  f = sin(x1)*exp((1-cos(x2))^2) + cos(x2)*exp((1-sin(x1))^2) + (x1-x2)^2
  return(f) } 

birdgrad <- function(par)  {
  x1 = par[1];  x2 = par[2]
  g = c(0, 0)
  g[1] = cos(x1)*exp((1-cos(x2))^2) - 2*cos(x2)*exp((1-sin(x1))^2)*(1-sin(x1))*cos(x1) + 2*(x1-x2)
  g[2] = 2*sin(x1)*exp((1-cos(x2))^2)*(1-cos(x2))*sin(x2) - sin(x2)*exp((1-sin(x1))^2) + 2*(x2-x1)
  return(g) } 
out <- oscarsQN(bird, birdgrad, 2, -10, 50)

# Hosaki function with global minimum of -2.3458 at (4,2) and one local minimum
hosaki <- function(par)  {
  x = par[1]
  y = par[2]
  f = (1 - 8*x + 7*x^2 - (7/3)*x^3 + (1/4)*x^4)*y*y*exp(-y)
  return(f) }

hosakigrad <- function(par)  {
  x = par[1]
  y = par[2]
  g = c(0, 0)
  g[1] = (-8 + 14*x - 7*x^2 + x^3)*y*y*exp(-y)
  g[2] = (1 - 8*x + 7*x^2 - (7/3)*x^3 + (1/4)*x^4)*(2-y)*y*exp(-y)
  return(g) }
out <- oscarsQN(hosaki, hosakigrad, 2, 0, upr = c(5,6))

# Rosenbrocks "banana" function with global minimum of zero at (a, a^2)
rosenbrock <- function(par, a = 1, b = 100) {
  f = (a - par[1])^2 + b*(par[2] - par[1]^2)^2
  return(f)  }

rosenbrockgrad <- function(par, a = 1, b = 100) {
  g = c(0, 0)
  g[1] = -2*(a - par[1]) + 2*b*(par[2] - par[1]^2)*(-2*par[1])
  g[2] = 2*b*(par[2] - par[1]^2)
  return(g)  }
out <- oscarsQN(rosenbrock, rosenbrockgrad, 2, -3, 3, a = 0.5)

# Schwefel function with global min of -418.9829n at x_i = 420.97...
# in n dimensions, where n is arbitrary.
schwefel <- function(par) {
  f = - sum(par*sin(sqrt(abs(par))))
  return(f)  }

schwefelgrad <- function(par) {
  rootpar = sqrt(abs(par))
  g = -sin(rootpar) - 0.5*rootpar*cos(rootpar)
  return(g)  }
out <- oscarsQN(schwefel, schwefelgrad, n = 3, -500, 500)
# This problem is solved in n = 3 dimensions here.

# vardim function with global min of 0 at par[i] = 1 in n dimensions.
vardim <- function(par) {
  n = length(par)
  temp = c(1:n)
  fn1 = sum(temp*(par-1))
  f = sum((par-1)^2) + fn1^2 + fn1^4
  return(f)   }

vardimgrad <- function(par) {
  n = length(par)
  temp = c(1:n)
  fn1 = sum(temp*(par-1))
  g = 2*(par-1) + (2*fn1 + 4*fn1^3)*temp
  return(g)  }
out <- oscarsQN(vardim, vardimgrad, n = 5, 0, 2.7182818)
# dixon function with global min of 0 in n dimensions at par[i] = 1.
dixon <- function(par) {
  n = length(par)
  xlo = par[1:n-1]
  xhi = par[2:n]
  f = (1-par[1])^2 + (1-par[n])^2 + sum( (xlo^2 - xhi)^2 )
  return(f)   }

dixongrad <- function(par) {
  n = length(par)
  x = par
  g = rep(0, times = n)
  g[1] = -2*(1-x[1]) + 2*(x[1]^2 - x[2])*2*x[1]
  jk = 2
  while (jk < n)  {
    #cat(sprintf("j = %i \n",jk))
    g[jk] = 2*(x[jk]^2 - x[jk+1])*2*x[jk] - 2*(x[jk-1]^2 - x[jk])
    jk = jk+1
  }
  g[n] = -2*(1-x[n]) + 2*(x[n-1]^2 - x[n])*(-1)
  return(g)  }
out <- oscarsQN(dixon, dixongrad, n = 4, -2, 2)

Control parameters for oscarsQN routine

Description

Provides control over oscarsQN parameters, such as number of iterations, tolerance, etc.

Usage

oscarsQN.control(
  nfmax = 50000,
  infol = 1,
  DoMax = FALSE,
  fTol = 1e-05,
  xTol = 1e-08,
  kktTol = 1e-05,
  kktstopcount = 3,
  fTarget = NULL,
  CompareGrad = FALSE
)

Arguments

Details

A subset of parameters can be specified. All non-specified parameters revert to their defaults. No parameter abbreviations.

Value

A named list of control parameters for oscars.

Examples

oscarsQN.control()  # default values
oscarsQN.control(nfmax = 100000) # bump iteration budget
oscarsQN.control(xTol = 10*oscars.control()$xTol) # increase xTol

Parallel OSCARS-II bound constrained global optimization

Description

poscars is a parallel wrapper around oscars that runs several independent OSCARS-II searches at the same time, one per processor core, and returning the best result found.

Usage

poscars(
  fname,
  n,
  lwr,
  upr,
  ...,
  start = NULL,
  controls = oscars.control(),
  ncores = 2,
  divide.budget = TRUE,
  seed = NULL,
  cl = NULL
)

Arguments

Details

OSCARS-II is a stochastic direct search: each run draws random sample points inside a sequence of nested, shrinking boxes and periodically restarts from a random or the best known point (see oscars for full details). Each run depends on its own chain of random draws and the inner search loop cannot be split across cores. However, independent whole runs are possible and poscars launches ncores independent runs, each seeded with its own reproducible random number stream, and keeps whichever run finds the best objective value.

Each worker run stops using the ordinary oscars stopping rules. OSCARS stopping rules are governed by tolerances fTol and xTol (see oscars.control).

By default, poscars divides the total evaluation budget nfmax evenly among the ncores workers (see divide.budget). This keeps the total number of function evaluations roughly the same as a single serial oscars call, but should execute faster by roughly ncores times. Set divide.budget = FALSE to give every worker the full budget, which does more total work but improves the chance of locating the global optimum.

The objective function fname, its ... arguments, the bounds and the controls are sent to each worker. Any objects that fname uses from the global workspace are not automatically exported to the workers. To be safe, make fname self-contained or pass everything it needs through ....

Value

A list of class "oscars" containing results of the optimization. This list consists of the following components:

• par: vector containing the best known parameters

• value: The minimized (or maximized) function value

• evaluations: The number of function evaluations used

• cycles: The number of cycles used

• convergence: 0 if the function and parameter tolerances have been reached; 1 if tolerances have not been reached but function evaluation budget has been exhausted; 2 if bounds are inconsistent.

• message: A text string explaining the value in convergence.

• controls: The values of the controls provided to oscars.

• ncores: The number of cores used.

• all.values: A vector containing the best values returned by every worker.

See Also

oscars for the underlying algorithm and the meaning of the return fields; oscars.control for the control parameters; makeCluster for supplying your own cluster.

Examples

# Per CRAN policies, these examples run on only 2 cores.
# Set ncores = Inf to utilize all cores.
# Setting ncores = parallel::detectCores()-1 is a good choice.

# Camel function with global minima of f = -1.0316 at
# (0.0898, 0.7127) and (0.0898, -0.7127) plus four other local minima.
camel <- function(par) {
  x <- par[1]
  y <- par[2]
  4*x^2 - 2.1*x^4 + (1/3)*x^6 + x*y + 4*(y^4 - y^2)
}
# Run four independent searches in parallel and keep the best.
out <- poscars(camel, n = 2, lwr = c(-5, -5), upr = c(5, 5), ncores = 2)
out

# Reproducible parallel run via the seed argument.
out1 <- poscars(camel, 2, -5, 5, ncores = 2, seed = 42)
out2 <- poscars(camel, 2, -5, 5, ncores = 2, seed = 42)
identical(out1$value, out2$value)

# Passing extra arguments to the objective function.
# Rosenbrock's "banana" function, global minimum 0 at (a, a^2).
rosenbrock <- function(par, a = 1, b = 100) {
  (a - par[1])^2 + b*(par[2] - par[1]^2)^2
}
out <- poscars(rosenbrock, 2, -3, 3, a = 0.5, ncores = 2)

# Best-of-ncores multi-start: give every worker the full budget instead
# of dividing it, trading more total work for a more thorough search.
out <- poscars(camel, 2, -5, 5, ncores = 2, divide.budget = FALSE,
               controls = oscars.control(nfmax = 20000, infol = 0))

# Reuse a single cluster across several calls to avoid start-up cost.
cl <- parallel::makeCluster(2)
o1 <- poscars(camel, 2, -5, 5, cl = cl)
o2 <- poscars(rosenbrock, 2, -3, 3, a = 0.5, cl = cl)
parallel::stopCluster(cl)

Print method for 'oscars' objects

Description

Prints an 'oscars' object showing minimized (or maximized) parameters and the optimization message.

Usage

## S3 method for class 'oscars'
print(x, ...)

Arguments

Value

No return value, called for side effects. Technically, NULL is returned invisibly.

See Also

oscars

Examples

# Branins camel function with global minimum of f = -1.0316 at
# (0.0898,0.7127) and (0.0898,-0.7127) with four other local minimizers
camel <- function(par) {
  x = par[1]
  y = par[2]
  f = 4*x^2 - 2.1*x^4 + (1/3)*x^6 + x*y + 4*(y^4-y^2)
  return(f) }
out <- oscars(camel, n = 2, lwr = c(-5,-5), upr = c(5,5))
out

Print method for 'oscars' objects

Description

Prints an 'oscars' object showing minimized (or maximized) parameters and the optimization message.

Usage

## S3 method for class 'oscarsQN'
print(x, ...)

Arguments

Value

No return value, called for side effects. Technically, NULL is returned invisibly.

See Also

oscars

Examples

# Hosaki function with global minimum of -2.3458 at (4,2) and one local minimum
hosaki <- function(par)  {
  x = par[1]
  y = par[2]
  f = (1 - 8*x + 7*x^2 - (7/3)*x^3 + (1/4)*x^4)*y*y*exp(-y)
  return(f) }

hosakigrad <- function(par)  {
  x = par[1]
  y = par[2]
  g = c(0, 0)
  g[1] = (-8 + 14*x - 7*x^2 + x^3)*y*y*exp(-y)
  g[2] = (1 - 8*x + 7*x^2 - (7/3)*x^3 + (1/4)*x^4)*(2-y)*y*exp(-y)
  return(g) }
out <- oscarsQN(hosaki, hosakigrad, 2, 0, upr = c(5,6))
out

Summary method for 'oscars' objects

Description

Summarizes an 'oscars' object. Shows an 'oscars' object's minimized (or maximized) parameters, optimization message, iterations, etc..

Usage

## S3 method for class 'oscars'
summary(object, ...)

Arguments

Value

No return value, called for side effects. Technically, NULL is returned invisibly.

See Also

oscars

Examples

# Branins camel function with global minimum of f = -1.0316 at
# (0.0898,0.7127) and (0.0898,-0.7127) with four other local minimizers
camel <- function(par) {
  x = par[1]
  y = par[2]
  f = 4*x^2 - 2.1*x^4 + (1/3)*x^6 + x*y + 4*(y^4-y^2)
  return(f) }
out <- oscars(camel, n = 2, lwr = c(-5,-5), upr = c(5,5))
summary(out)

Summary method for 'oscarsQN' objects

Description

Summarizes an 'oscarsQN' object. Shows an 'oscarsQN' object's minimized (or maximized) parameters, optimization message, iterations, etc..

Usage

## S3 method for class 'oscarsQN'
summary(object, ...)

Arguments

Value

No return value, called for side effects. Technically, NULL is returned invisibly.

See Also

oscarsQN

Examples

# Hosaki function with global minimum of -2.3458 at (4,2) and one local minimum
hosaki <- function(par)  {
  x = par[1]
  y = par[2]
  f = (1 - 8*x + 7*x^2 - (7/3)*x^3 + (1/4)*x^4)*y*y*exp(-y)
  return(f) }

hosakigrad <- function(par)  {
  x = par[1]
  y = par[2]
  g = c(0, 0)
  g[1] = (-8 + 14*x - 7*x^2 + x^3)*y*y*exp(-y)
  g[2] = (1 - 8*x + 7*x^2 - (7/3)*x^3 + (1/4)*x^4)*(2-y)*y*exp(-y)
  return(g) }
out <- oscarsQN(hosaki, hosakigrad, 2, 0, upr = c(5,6))
summary(out)