Get Aggregated Performance Values by Generation

Get evaluated performance values from an OptimInstance aggregated for each generation. This may either concern all individuals that were alive at the end of a given generation (survivors_only TRUE) or at any point during a generation (survivors_only FALSE).

The result is a single data.table object with a dob column indicating the generation, as well as one column for each aggregations entry crossed with each objective of inst.

See mies_generation_apply() on how to apply functions to entire fitness-matrices, not only individual objectives.

mies_aggregate_generations(
  inst,
  objectives = inst$archive$codomain$ids(),
  aggregations = list(min = min, mean = mean, max = max, median = stats::median, size =
    length),
  as_fitnesses = TRUE,
  survivors_only = TRUE,
  condition_on_budget_id = NULL
)

Arguments

inst: (OptimInstance)
Optimization instance to evaluate.
objectives: (character)
Objectives for which to calculate aggregates. Must be a subset of the codomain elements of inst, but when as_fitnesses is TRUE, elements that are neither being minimized nor maximized are ignored.
aggregations: (named list of function)
List containing aggregation functions to be evaluated on a vector of objective falues for each generation. These functions should take a single argument and return a scalar value.
as_fitnesses: (logical(1))
Whether to transform performance values into "fitness" values that are always to be maximized. This means that values that objectives that should originally be minimized are multiplied with -1, and that parts of the objective codomain that are neither being minimized nor maximized are dropped. Default TRUE.
survivors_only: (logical(1))
Whether to ignore configurations that have "eol" set to the given generation, i.e. individuals that were killed during that generation. When this is TRUE (default), then only individuals that are alive at the end of a generation are considered; otherwise all individuals alive at any point of a generation are considered. If it is TRUE, this leads to individuals that have "dob" == "eol" being ignored.
condition_on_budget_id: (character(1) | NULL)
Budget component when doing multi-fidelity optimization. When this is given, then for each generation, only individuals with the highest value for this component are considered. If survivors_only is TRUE, this means the highest value of all survivors of a given generation, if it is FALSE, then it is the highest value of all individuals alive at any point of a generation. To ignore possible budget-parameters, set this to NULL (default). This is inparticular necessary when fidelity is not monotonically increasing (e.g. if it is categorical).

Value

a data.table with the column "dob", indicating the generation, as well as further columns named by the items in aggregations. There is more on element in objectives

(or more than one element not being minimized/maximized when as_fitnesses is TRUE), then columns are named <aggregations element name>.<objective name>. Otherwise, they are named by <aggregations element name> only. To get a guarantee that elements are only named after elements in aggregations, set objectives

to a length 1 character.

Examples

library("bbotk")
lgr::threshold("warn")

# Define the objective to optimize
objective <- ObjectiveRFun$new(
  fun = function(xs) {
    z <- 10 - exp(-xs$x^2 - xs$y^2) + 2 * exp(-(2 - xs$x)^2 - (2 - xs$y)^2)
    list(Obj = z)
  },
  domain = ps(x = p_dbl(-2, 4), y = p_dbl(-2, 4)),
  codomain = ps(Obj = p_dbl(tags = "minimize"))
)

oi <- OptimInstanceSingleCrit$new(objective,
  terminator = trm("evals", n_evals = 6)
)

op <- opt("mies",
  lambda = 2, mu = 2,
  mutator = mut("gauss", sdev = 0.1),
  recombinator = rec("xounif"),
  parent_selector = sel("best")
)
set.seed(1)
op$optimize(oi)
#>            x       y  x_domain      Obj
#>        <num>   <num>    <list>    <num>
#> 1: -0.406948 1.43712 <list[2]> 9.897011

# negates objectives that are minimized:
mies_aggregate_generations(oi)
#>      dob        min      mean       max    median  size
#>    <int>      <num>     <num>     <num>     <num> <int>
#> 1:     1 -10.010770 -9.953890 -9.897011 -9.953890     2
#> 2:     2 -10.000021 -9.948516 -9.897011 -9.948516     2
#> 3:     3  -9.944994 -9.921002 -9.897011 -9.921002     2

# silly aggregation: first element
mies_aggregate_generations(oi, aggregations = list(first = function(x) x[1]))
#>      dob     first
#>    <int>     <num>
#> 1:     1 -9.897011
#> 2:     2 -9.897011
#> 3:     3 -9.897011

# real objective values:
mies_aggregate_generations(oi, as_fitnesses = FALSE)
#>      dob      min     mean       max   median  size
#>    <int>    <num>    <num>     <num>    <num> <int>
#> 1:     1 9.897011 9.953890 10.010770 9.953890     2
#> 2:     2 9.897011 9.948516 10.000021 9.948516     2
#> 3:     3 9.897011 9.921002  9.944994 9.921002     2

# Individuals that died are included:
mies_aggregate_generations(oi, survivors_only = FALSE)
#>      dob       min       mean       max     median  size
#>    <int>     <num>      <num>     <num>      <num> <int>
#> 1:     1 -10.01077  -9.953890 -9.897011  -9.953890     2
#> 2:     2 -10.30388 -10.052922 -9.897011 -10.005396     4
#> 3:     3 -10.00124  -9.960816 -9.897011  -9.972507     4

Arguments

Value

See also

Examples