Offline Surrogate-Performance Replay — replay_surrogate

Reconstructs the surrogate-performance learning curve of a finished active-learning run, offline, from its bbotk::ArchiveBatch. For every batch the surrogate is refit on the archive data available up to that batch and scored on a held-out regression task, yielding one performance value per batch and measure. The result is assembled into a TaskLCE, the same shape produced online by CallbackSurrogatePerformance.

This is the offline twin of CallbackSurrogatePerformance: the callback scores the surrogate the optimizer maintains during the run, whereas this function replays the run from a stored archive, refitting learner itself.

Usage

replay_surrogate_performance(
  archive,
  learner,
  task,
  measures = list(msr("regr.rsq"), msr("regr.mae")),
  measure = NULL,
  pool = NULL,
  link = "identity",
  id = "surrogate_performance",
  label = NA_character_
)

Arguments

archive: (bbotk::ArchiveBatch)
Archive of a finished single-target run.
learner: (mlr3::LearnerRegr | mlr3mbo::SurrogateLearner)
Surrogate model refit per batch. A plain regression learner is wrapped in a mlr3mbo::SurrogateLearner so the fit / predict / output-transform path matches a live run; a surrogate is used as given (cloned). Either way the object is cloned, so the caller's model is left untrained.
task: (mlr3::TaskRegr)
Held-out regression task to score on. Must contain the archive's search-space columns as features.
measures: (list() of mlr3::Measure)
Regression measures. Named entries use their names as performance-column names; unnamed entries use the measure id.
measure: (character(1) | NULL)
Name of the measure to use as the TaskLCE target. When NULL (default) and a single measure is given, that measure is used.
pool: (data.table::data.table | NULL)
Finite candidate pool of the originating run, carried into the resulting TaskLCE for pool-based replay. Its columns must be the archive's search-space ids. NULL (default) for continuous runs.
link: (character(1))
Name of the predictive lce_link for the resulting TaskLCE. "identity" by default; pass e.g. "log" for a non-negative loss metric (see lce_link_from_range).
id: (character(1))
Task id.
label: (character(1))
Optional task label.

Value

TaskLCE.