milia.benchmark.runs#

Run folders: <root>/<experiment>/runs/<timestamp>-<hash>-<commit>/ with a latest link.

Module Attributes

ROOT

Environment variable overriding the results root.

Functions

commit()

Short git commit of the working directory.

latest(experiment[, base])

The most recent run folder of an experiment.

root()

The results root: $MILIA_RESULTS, or ./results.

start(experiment[, settings, base])

Create a new run folder, write its meta.json and point latest at it.

milia.benchmark.runs.ROOT = 'MILIA_RESULTS'#

Environment variable overriding the results root.

milia.benchmark.runs.root()#

The results root: $MILIA_RESULTS, or ./results.

Return type:

Path

Returns:

The root directory (not created).

Examples

>>> import os
>>> os.environ["MILIA_RESULTS"] = "/tmp/out"
>>> str(root())
'/tmp/out'
>>> del os.environ["MILIA_RESULTS"]
milia.benchmark.runs.commit()#

Short git commit of the working directory.

Return type:

str

Returns:

The commit hash, or "nogit" outside a repository.

Examples

>>> isinstance(commit(), str)
True
milia.benchmark.runs.start(experiment, settings=None, base=None)#

Create a new run folder, write its meta.json and point latest at it.

The folder name joins a UTC timestamp, a hash of the settings and the git commit, so two runs never collide and a run states what produced it.

Parameters:
  • experiment (str) – Experiment name (a folder under the root).

  • settings (dict[str, Any] | None) – JSON-serialisable run settings, stored in meta.json.

  • base (Path | None) – Results root; root() when omitted.

Return type:

Path

Returns:

The new run folder.

Examples

>>> import tempfile
>>> from pathlib import Path
>>> folder = start("detectors", {"alpha": 0.1}, base=Path(tempfile.mkdtemp()))
>>> (folder / "meta.json").exists(), folder.parent.name
(True, 'runs')
milia.benchmark.runs.latest(experiment, base=None)#

The most recent run folder of an experiment.

Parameters:
  • experiment (str) – Experiment name.

  • base (Path | None) – Results root; root() when omitted.

Return type:

Path

Returns:

The resolved run folder.

Raises:

FileNotFoundError – If the experiment has no runs.

Examples

>>> import tempfile
>>> from pathlib import Path
>>> base = Path(tempfile.mkdtemp())
>>> folder = start("detectors", base=base)
>>> latest("detectors", base) == folder.resolve()
True