milia.anomaly.inject#

Synthetic anomalies to inject into otherwise normal profiles.

Real labelled anomalies are rare, so detectors are evaluated by injecting known faults into held-out normal days. Every injector takes one profile (a 1-D array of readings, typically one day) and a numpy generator, and returns a modified copy. Default ranges scale with the profile length and reproduce the anomaly chapter’s settings exactly for 48-step (half-hourly) days.

Module Attributes

TRIPS

Number of trips on a faulty day, and their length in steps, as [low, high) ranges.

CLOCKSHIFT

Offset of a meter clock that records UTC as local time in Victoria (AEST, UTC+10), in half-hours.

ADOPTIONS

a Level 1 or Level 2 EV charger, a PV array, or both.

LEVEL1

Power of a Level 1 charger (2.4 kW) relative to the Level 2 sessions (7 kW) of a session pool.

BEHAVIOURFAULTS

split into episodes, or moved in time.

METERFAULTS

Faults of a meter's readings.

INJECTIONS

Meter faults used by the anomaly chapter's benchmark.

BEHAVIOURS

Prosumer behaviours on net load (export negative).

Functions

adopt(cube, adopters, technologies, ...[, ...])

Add a technology to the readings of each adopter, from a given day to the end of the period.

behaviourfault(days, kind, fault, count, ...)

Inject a behaviour fault into randomly chosen days on which the behaviour occurs.

clockerror(days, day[, shift])

The readings a meter records for one day when its clock runs shift steps behind local time.

clockshift(profile[, generator, shift])

A day recorded by a meter whose clock runs shift steps behind local time, from the day's own readings.

drift(profile, rng[, factors])

Multiply by a ramp from 1 to a random end factor: calibration drift.

dropout(profile, rng[, lengths, latest])

Set a block of readings to zero: a communication dropout.

fragmented(profile, rng[, episodes])

Break the export (negative) readings into several short episodes at random times.

meterfault(cube, meters, fault, generator[, ...])

Inject one fault into the readings of chosen meters.

offhours(profile, rng[, shifts])

Move the export block to an unusual time of day.

shift(profile, rng, shape[, scales])

Add a scaled shape: a new behaviour such as an EV charger (positive) or PV (negative).

span(n, lengths, default)

Length range of an injected block, scaled to the profile length when not given.

spike(profile, rng[, factors, width])

Multiply a short block by a large factor: a spike.

stuck(profile, rng[, lengths, latest])

Hold the reading constant over a block: a stuck meter.

tripping(profile, generator[, trips])

An inverter that trips repeatedly within the export window of one day.

underreport(profile[, rng, fraction])

Scale readings down: a meter tampered to under-report.

milia.anomaly.inject.span(n, lengths, default)#

Length range of an injected block, scaled to the profile length when not given.

Parameters:
Return type:

tuple[int, int]

Returns:

(low, high) in steps.

Examples

>>> span(48, None, (1 / 8, 1 / 4))
(6, 12)
milia.anomaly.inject.stuck(profile, rng, lengths=None, latest=None)#

Hold the reading constant over a block: a stuck meter.

Parameters:
  • profile (ArrayLike) – Readings.

  • rng (Generator) – Random generator.

  • lengths (tuple[int, int] | None) – [low, high) block length; n/8 to n/4 by default.

  • latest (int | None) – Last allowed start (exclusive); 5n/6 by default.

Return type:

ndarray

Returns:

A modified copy of the profile.

Examples

>>> day = np.linspace(1.0, 2.0, 48)
>>> out = stuck(day, np.random.default_rng(0))
>>> int((np.diff(out) == 0).sum()) >= 5
True
milia.anomaly.inject.dropout(profile, rng, lengths=None, latest=None)#

Set a block of readings to zero: a communication dropout.

Parameters:
  • profile (ArrayLike) – Readings.

  • rng (Generator) – Random generator.

  • lengths (tuple[int, int] | None) – [low, high) block length; n/12 to 5n/24 by default.

  • latest (int | None) – Last allowed start (exclusive); 5n/6 by default.

Return type:

ndarray

Returns:

A modified copy of the profile.

Examples

>>> day = np.linspace(1.0, 2.0, 48)
>>> int((dropout(day, np.random.default_rng(0)) == 0).sum()) >= 4
True
milia.anomaly.inject.drift(profile, rng, factors=(1.4, 1.8))#

Multiply by a ramp from 1 to a random end factor: calibration drift.

Parameters:
Return type:

ndarray

Returns:

A modified copy of the profile.

Examples

>>> day = np.linspace(1.0, 2.0, 48)
>>> out = drift(day, np.random.default_rng(0))
>>> bool(out[0] == day[0] and out[-1] > 1.4 * day[-1] - 1e-9)
True
milia.anomaly.inject.spike(profile, rng, factors=(2.0, 4.0), width=1)#

Multiply a short block by a large factor: a spike.

Parameters:
  • profile (ArrayLike) – Readings.

  • rng (Generator) – Random generator.

  • factors (tuple[float, float]) – [low, high] range of the multiplier.

  • width (int) – Block length in steps.

Return type:

ndarray

Returns:

A modified copy of the profile.

Examples

>>> day = np.linspace(1.0, 2.0, 48)
>>> bool(spike(day, np.random.default_rng(0)).max() >= 2 * day.min())
True
milia.anomaly.inject.fragmented(profile, rng, episodes=(4, 7))#

Break the export (negative) readings into several short episodes at random times.

The exported values are split, in order, into non-overlapping episodes placed at random positions, so the total export is kept and only its timing changes. A profile without export is returned unchanged.

Parameters:
  • profile (ArrayLike) – Net-load readings, negative while exporting.

  • rng (Generator) – Random generator.

  • episodes (tuple[int, int]) – [low, high) number of episodes.

Return type:

ndarray

Returns:

A modified copy of the profile.

Examples

>>> net = np.ones(48)
>>> net[20:28] = -2.0
>>> out = fragmented(net, np.random.default_rng(1))
>>> float(out[out < 0].sum()), bool((out != net).any())
(-16.0, True)
milia.anomaly.inject.offhours(profile, rng, shifts=None)#

Move the export block to an unusual time of day.

Parameters:
  • profile (ArrayLike) – Readings, export negative.

  • rng (Generator) – Random generator.

  • shifts (tuple[int, int] | None) – [low, high) shift in steps; n/3 to n/2 by default.

Return type:

ndarray

Returns:

A modified copy of the profile.

Examples

>>> net = np.ones(48)
>>> net[20:28] = -2.0
>>> int(np.flatnonzero(offhours(net, np.random.default_rng(1)) < 0)[0]) != 20
True
milia.anomaly.inject.underreport(profile, rng=None, fraction=0.5)#

Scale readings down: a meter tampered to under-report.

Parameters:
  • profile (ArrayLike) – Readings.

  • rng (Generator | None) – Unused; present so every injector shares a signature.

  • fraction (float) – Share of consumption hidden.

Return type:

ndarray

Returns:

profile * (1 - fraction).

Examples

>>> underreport([2.0, 4.0], fraction=0.25).tolist()
[1.5, 3.0]
milia.anomaly.inject.shift(profile, rng, shape, scales=(1.0, 1.0))#

Add a scaled shape: a new behaviour such as an EV charger (positive) or PV (negative).

Parameters:
  • profile (ArrayLike) – Readings.

  • rng (Generator) – Random generator.

  • shape (ArrayLike) – Additive shape, same length as profile.

  • scales (tuple[float, float]) – [low, high] uniform range of the shape’s multiplier.

Return type:

ndarray

Returns:

profile + factor * shape.

Examples

>>> shift(np.ones(3), np.random.default_rng(0), [0.0, 7.0, 0.0], scales=(1.0, 1.0)).tolist()
[1.0, 8.0, 1.0]
milia.anomaly.inject.TRIPS: tuple[tuple[int, int], tuple[int, int]] = ((2, 5), (1, 4))#

Number of trips on a faulty day, and their length in steps, as [low, high) ranges.

milia.anomaly.inject.CLOCKSHIFT = 20#

Offset of a meter clock that records UTC as local time in Victoria (AEST, UTC+10), in half-hours.

milia.anomaly.inject.ADOPTIONS: tuple[str, ...] = ('ev1', 'ev2', 'pv', 'pvev')#

a Level 1 or Level 2 EV charger, a PV array, or both.

Type:

Technologies an adopter installs

milia.anomaly.inject.LEVEL1 = 0.34285714285714286#

Power of a Level 1 charger (2.4 kW) relative to the Level 2 sessions (7 kW) of a session pool.

milia.anomaly.inject.tripping(profile, generator, trips=((2, 5), (1, 4)))#

An inverter that trips repeatedly within the export window of one day.

On its over-voltage or anti-islanding protection the inverter stops exporting for short spells. During each spell the PV output is lost, so the meter records the household’s demand, taken as the median of the day’s import readings, since the demand under the array is not observed. Unlike fragmented(), the energy of the spells is lost, not moved, and no export appears outside the export window.

Parameters:
  • profile (ArrayLike) – Net-load readings of one day, negative while exporting.

  • generator (Generator) – Random generator.

  • trips (tuple[tuple[int, int], tuple[int, int]]) – [low, high) number of trips and [low, high) length of each in steps.

Return type:

ndarray

Returns:

A modified copy; a day without export is returned unchanged.

Examples

>>> day = np.full(48, 0.5)
>>> day[20:30] = -2.0
>>> tripped = tripping(day, np.random.default_rng(0))
>>> bool((tripped[:20] == day[:20]).all() and (tripped[20:30] > 0).any())
True
milia.anomaly.inject.clockerror(days, day, shift=20)#

The readings a meter records for one day when its clock runs shift steps behind local time.

A meter that records UTC as local time labels every reading early by the time-zone offset, so the readings of a recorded day are those of local times shift steps later, partly from the following day. The fault needs consecutive days, so it is not a one-day Injector.

Parameters:
  • days (ArrayLike) – Readings of consecutive days, shape (days, steps).

  • day (int) – The day whose recorded readings are replaced; it must not be the last day.

  • shift (int) – Offset in steps.

Return type:

ndarray

Returns:

The readings recorded for day.

Raises:

ValueError – If day is the last day, whose later readings are not available.

Examples

>>> days = np.arange(12.0).reshape(3, 4)
>>> clockerror(days, 0, shift=1).tolist()
[1.0, 2.0, 3.0, 4.0]
milia.anomaly.inject.clockshift(profile, generator=None, shift=20)#

A day recorded by a meter whose clock runs shift steps behind local time, from the day’s own readings.

clockerror() needs the readings of the following day. Where days are evaluated one at a time, as by selfbaseline(), the recorded day is approximated by rolling the day’s readings, which is exact when consecutive days are alike: a midday export then appears shift steps earlier, in the night, while the energy and the shape of the day are unchanged.

Parameters:
  • profile (ArrayLike) – Readings of one day.

  • generator (Generator | None) – Unused; present so every injector shares a signature.

  • shift (int) – Offset in steps; 20 half-hours is ten hours, the offset of UTC from the time of Victoria.

Return type:

ndarray

Returns:

The recorded day.

Examples

>>> clockshift(np.arange(6.0), shift=2).tolist()
[2.0, 3.0, 4.0, 5.0, 0.0, 1.0]
milia.anomaly.inject.adopt(cube, adopters, technologies, generator, pv, sessions, kwp=(3.0, 8.0), onset=0)#

Add a technology to the readings of each adopter, from a given day to the end of the period.

PV subtracts an array of random size times the output per kWp, so the readings become a net load that is negative while the household exports. Level 2 charging adds a vehicle’s recorded sessions; Level 1 adds them scaled to 2.4 kW.

Parameters:
  • cube (ArrayLike) – Readings in kW, shape (meters, days, steps).

  • adopters (ArrayLike) – Indices of the adopting meters.

  • technologies (list[str]) – One of ADOPTIONS per adopter.

  • generator (Generator) – Random generator, for the vehicle and the array size.

  • pv (ArrayLike) – PV output per kWp, shape (days, steps).

  • sessions (ArrayLike) – Recorded charging of a session pool in kW, shape (vehicles, days, steps).

  • kwp (tuple[float, float]) – Range of array sizes in kWp.

  • onset (int) – First day with the technology installed.

Return type:

ndarray

Returns:

A modified copy of the readings.

Raises:

ValueError – If a technology is not one of ADOPTIONS.

Examples

>>> cube, pool = np.ones((2, 3, 4)), np.full((1, 3, 4), 7.0)
>>> adopted = adopt(cube, [0], ["ev2"], np.random.default_rng(0), np.zeros((3, 4)), pool, onset=1)
>>> adopted[0, 0].tolist(), adopted[0, 1].tolist()
([1.0, 1.0, 1.0, 1.0], [8.0, 8.0, 8.0, 8.0])
milia.anomaly.inject.BEHAVIOURFAULTS: tuple[str, ...] = ('fragmented', 'offhours')#

split into episodes, or moved in time.

Type:

Faults of a day’s behaviour

milia.anomaly.inject.METERFAULTS: tuple[str, ...] = ('stuck', 'dropout', 'underreport')#

Faults of a meter’s readings.

milia.anomaly.inject.behaviourfault(days, kind, fault, count, generator)#

Inject a behaviour fault into randomly chosen days on which the behaviour occurs.

For export (net load, negative while exporting), "fragmented" is tripping() and "offhours" is clockerror(), the physical faults of an inverter and of a meter clock. For charging (power drawn by a charger), they split a session into fragments (fragmented()) or move it in time (offhours()), as a faulty charger or a misconfigured timer would.

Parameters:
  • days (ArrayLike) – Readings of consecutive days, shape (days, steps).

  • kind (str) – "export" or "charging".

  • fault (str) – One of BEHAVIOURFAULTS.

  • count (int) – Number of days to corrupt; fewer if fewer days show the behaviour.

  • generator (Generator) – Random generator.

Return type:

tuple[ndarray, ndarray]

Returns:

The corrupted copy and a boolean label per day.

Raises:

ValueError – If kind or fault is unknown.

Examples

>>> days = np.full((6, 48), 0.5)
>>> days[:, 20:30] = -2.0
>>> corrupted, labels = behaviourfault(days, "export", "offhours", 2, np.random.default_rng(0))
>>> int(labels.sum()), bool((corrupted[labels][:, :8] < 0).any())
(2, True)
milia.anomaly.inject.meterfault(cube, meters, fault, generator, days=30, resolution=0.001, interval=0.5)#

Inject one fault into the readings of chosen meters.

A stuck register or a dropout affects days random days of each meter. Under-recording hides half of every reading: the stored readings are already rounded to the register’s resolution, so the energy lost to that rounding is restored with a uniform dither within one step before the share is hidden and the register rounds again; halving the rounded readings directly would create ties that no real register produces.

Parameters:
  • cube (ArrayLike) – Power in kW, shape (meters, days, steps).

  • meters (ArrayLike) – Indices of the meters to corrupt.

  • fault (str) – One of METERFAULTS.

  • generator (Generator) – Random generator.

  • days (int) – Days corrupted per meter for a stuck register or a dropout.

  • resolution (float) – The register’s resolution in kWh.

  • interval (float) – Length of a step in hours.

Return type:

ndarray

Returns:

A corrupted copy of the cube.

Raises:

ValueError – If fault is unknown.

Examples

>>> cube = np.full((2, 40, 48), 0.6)
>>> faulty = meterfault(cube, [1], "underreport", np.random.default_rng(0))
>>> round(float(faulty[0].mean()), 2), round(float(faulty[1].mean()), 2)
(0.6, 0.3)
milia.anomaly.inject.INJECTIONS: dict[str, Callable[..., ndarray]] = {'drift': <function drift>, 'dropout': <function dropout>, 'stuck': <function stuck>}#

Meter faults used by the anomaly chapter’s benchmark.

milia.anomaly.inject.BEHAVIOURS: dict[str, Callable[..., ndarray]] = {'fragmented': <function fragmented>, 'offhours': <function offhours>}#

Prosumer behaviours on net load (export negative).