API Reference

This page provides the autogenerated API reference for the public modules of pyOMA-Monitoring. Site-specific modules (site_tower.py and similar) are intentionally omitted from the public reference; see Configuration Guide for the template-based guide to writing one.

monitoring — engine functions

A data-organization, -storage and analysis scheme for long-term tower monitoring

store binary files on hard disk time synchronization must be achieved prior to analysis:

cases

pc/controller is ahead of controller/pc daylight saving clock of pc/controller changed available information:

timestamp controller, → Reference, because it does not change often and there is no daylight saving, though it may not relate to the “real time” timestamp pc, filetime controller, filetime pc

correct (and synchronize) files and store as binary on hard disk store results in a database structure and link to the file on disk; preferred data-organization: using different averages 1m, 10m and 1 hour should be possible

→ create multiple databases

Coordinates: time (synchronized to controller time but in local timezone (w/ daylight saving), channels Variables:

File Information: file_name (‘time’,) file_size (‘time’,) file_time (‘time’,) start_time (‘time’,)

Content Information: num_channels (‘time’,) units (‘channels’,) sample_rate (‘time’,) duration (‘time’,) (in seconds) length (‘time’,) (in samples) error (‘time’, ‘channels’)

Statistical Description of the recorded signal: mean (‘time’, ‘channels’) min (‘time’, ‘channels’) max (‘time’, ‘channels’) var (‘time’, ‘channels’) skewness (‘time’, ‘channels’) kurtosis (‘time’, ‘channels’) q05 (‘time’, ‘channels’) q50 (‘time’, ‘channels’) q95 (‘time’, ‘channels’) rms (‘time’, ‘channels’)

Coordinates: time, channels Variables:

Content Information: num_channels (‘time’,) sample_rate (‘time’,) length (‘time’,) error (‘time’, ‘channels’)

Statistical Description of the sliced and processed signal: mean (‘time’, ‘channels’) min (‘time’, ‘channels’) max (‘time’, ‘channels’) var (‘time’, ‘channels’) skewness (‘time’, ‘channels’) kurtosis (‘time’, ‘channels’) q05 (‘time’, ‘channels’) q50 (‘time’, ‘channels’) q95 (‘time’, ‘channels’) rms (‘time’, ‘channels’)

Coordinates: time, channels, modes Variables:

Content information of the sliced signal: num_channels (‘time’,) sample_rate (‘time’,) length (‘time’,) error (‘time’, ‘channels’)

Operational Modal Analysis num_modes (‘time’,) frequencies (‘time’, ‘modes’) damping (‘time’, ‘modes’) model_orders (‘time’, ‘modes’) std_damping (‘time’, ‘modes’) std_frequencies (‘time’, ‘modes’) var_svd_psd (‘time’, ‘channels’) modeshapes (‘time’, ‘modes’, ‘channels’) MPC (‘time’, ‘modes’) MPD (‘time’, ‘modes’)

Statistical description of the singular value PSD: max_svd_psd (‘time’, ‘channels’) mean_svd_psd (‘time’, ‘channels’) min_svd_psd (‘time’, ‘channels’) skewness_svd_psd (‘time’, ‘channels’) kurtosis_svd_psd (‘time’, ‘channels’) q05_svd_psd (‘time’, ‘channels’) q50_svd_psd (‘time’, ‘channels’) q95_svd_psd (‘time’, ‘channels’) rms_svd_psd (‘time’, ‘channels’) energy_svd_psd (‘time’, ‘channels’)

Statistical Description of the sliced and processed signal: mean (‘time’, ‘channels’) min (‘time’, ‘channels’) max (‘time’, ‘channels’) var (‘time’, ‘channels’) skewness (‘time’, ‘channels’) kurtosis (‘time’, ‘channels’) q05 (‘time’, ‘channels’) q50 (‘time’, ‘channels’) q95 (‘time’, ‘channels’) rms (‘time’, ‘channels’) _____________________________________________________________________

class monitoring.Site(name: str, transforms: Dict[str, Callable], setup_prep: Dict[str, Callable], error_rules: Dict[str, dict], sync_policy: Callable, modal_bands: List[tuple], file_list_fn: Callable, channel_mean_fn: Callable | None, preproc_channels: Dict[str, list], db_root_path: str, slice_root_path: str, modal_conf_dir: str, file_root_path: str, origins: Dict[str, str], subpaths: Dict[str, str], all_channels: Dict[str, list], optional_channels: Dict[str, list], dtstarts: Dict[str, object], ranges: Dict[str, tuple])

Bases: object

Contract between the generic engine and a site-specific implementation.

All callables are pure functions with no side-effects on the engine state. Path / channel / range fields carry site-specific configuration that was previously read directly from config.py by the engine.

all_channels: Dict[str, list]

channel_mean_fn: Callable | None

db_root_path: str

dtstarts: Dict[str, object]

error_rules: Dict[str, dict]

file_list_fn: Callable

file_root_path: str

modal_bands: List[tuple]

modal_conf_dir: str

name: str

optional_channels: Dict[str, list]

origins: Dict[str, str]

preproc_channels: Dict[str, list]

ranges: Dict[str, tuple]

setup_prep: Dict[str, Callable]

slice_root_path: str

subpaths: Dict[str, str]

sync_policy: Callable

transforms: Dict[str, Callable]

monitoring.close_to_utc_transition(file_time, close_hours=3)

Return True when file_time is near a DST transition.

Thin wrapper around TimeConvention.is_near_dst_transition() that also emits an INFO log message when the guard fires.

Parameters:

• file_time (datetime.datetime) – The timestamp to check.

• close_hours (int, optional) – Half-width of the exclusion window in hours. Defaults to 3.

Returns:

True when file_time is within ±*close_hours* of any DST transition in the project timezone.

Return type:

bool

monitoring.compute_gap_lengths(file_info)

check file_gaps (assumes files are sorted in time) this function only works reliably if all files have been read in before, therefore it can not be pre-computed and is re-computed every time the script is run

in Peaks_*, when the file compression script starts, all data written afterwards to the currently active file was lost therefore we have a gap of some minutes every night around 1 AM CET/CEST other gaps may be due to pc restarts, controller restarts, …

Peaks_* files have to be interleaved sometime, when the recording stopped eg in channel 2 and continued in the next file in channel 3 then there is a gap of (-1) sample which is removed during interleaving

monitoring.create_file_info(origin: str, chunksize: int = 50, skip_existing: bool = True, **kwargs)

file_name, file_size, file_creation_time, num_channels, headers, units, sample_rate, type (qstation, labview), start_time,<- 1st run length (in samples), per channel: {errors, mean, min, max, var, skewness, kurtosis, q05, q95, q50, rms} <- 1st run next_file, gap_length (in_samples) <- 2nd run

monitoring.create_modal_results(quantity: str, duration: Timedelta, stats: Dataset, chunksize: int = 2, skip_existing: bool = True, check_errors: bool = True, filter_errors: bool = True, **kwargs)

Run VarSSI-Ref modal analysis for every time step in stats.

For each time step that is not already in the master database the function calls, in order:

1. get_slice_preprocessed() — bandpass-filter and decimate the raw slice.

2. describe_stats() — compute per-channel signal statistics.

3. modal_analysis_single() — run VarSSI-Ref, cluster the stabilization diagram, and auto-select physical poles.

Results are accumulated in a per-process NetCDF file and then merged into the master database under MultiLock.

Parameters:

• quantity (str) – Quantity tag (e.g. 'accel', 'strain_rosettes').

• duration (pandas.Timedelta) – Analysis window length.

• stats (xarray.Dataset) – Statistics dataset (from get_stats()) providing the list of time steps to process and error flags to filter on.

• chunksize (int, optional) – Number of windows to accumulate before flushing the per-process NetCDF file to disk. Defaults to 2.

• skip_existing (bool, optional) – When True (default), skip time steps already present in the master database.

• check_errors (bool, optional) – When True (default), call check_and_mark_errors() on stats before iterating.

• filter_errors (bool, optional) – When True (default), skip time steps that have any error flag.

• **kwargs – Supported keys: dtstart, until, missing, num_workers, this_worker.

Returns:

The updated master modal-results dataset.

Return type:

xarray.Dataset

monitoring.create_stats(quantity: str, duration: Timedelta, file_info: Dataset, chunksize: int = 10, skip_existing: bool = False, **kwargs)

mean, min, max, var, skewness, kurtosis, q05, q95, q50, rms

monitoring.describe_stats(measurement, headers=None, quantity=None)

Compute per-channel descriptive statistics for a measurement array.

For each column of measurement the following statistics are computed: mean, min, max, variance, skewness, kurtosis, 5th/50th/95th percentiles, and RMS. An error flag is set when a channel contains only NaN values, has min == max (flatline), or violates the plausibility range defined in the active site’s ranges dict.

Parameters:

• measurement (numpy.ndarray, shape (n_samples, n_channels)) – Raw or preprocessed signal matrix. May contain NaN values.

• headers (list of str, optional) – Channel names in the same order as columns in measurement. Used to look up plausibility ranges and site-specific circular-mean functions. Defaults to empty strings.

• quantity (str, optional) – Quantity tag (e.g. 'accel') used to look up error_rules (kurtosis thresholds) in the active site.

Returns:

Keys: 'mean', 'min', 'max', 'var', 'skewness', 'kurtosis', 'q05', 'q50', 'q95', 'rms', 'error'. Each value is a 1-D numpy.ndarray of length n_channels.

Return type:

dict

monitoring.get_active_site() → Site | None

Return the currently active site, or None if none is set.

Returns:

The active Site instance, or None if set_active_site() has not been called yet.

Return type:

Site or None

monitoring.get_file_info(origin: str, create_new: bool = False, **kwargs)

Load or (re-)create the file-info database for origin.

Parameters:

• origin (str) – Origin tag identifying which sensor group to query.

• create_new (bool, optional) – When True, call create_file_info() instead of reading from disk. Defaults to False.

• **kwargs – Forwarded to create_file_info() when create_new is True.

Returns:

The file-info dataset with an additional gap_length variable (samples between consecutive files), or None if the on-disk database is newly created and empty.

Return type:

xarray.Dataset or None

monitoring.get_file_list(origin, reduced=False, file_info=None)

Return a list of raw data file paths for the given origin.

Delegates to the active site’s file_list_fn callback.

Parameters:

• origin (str) – Origin tag (e.g. 'accel', 'wind').

• reduced (bool, optional) – When True, the callback should return only files not already present in file_info (incremental update mode).

• file_info (xarray.Dataset or None, optional) – The current file-info database; used by the callback when reduced is True to filter already-indexed files.

Returns:

Absolute paths to the raw data files.

Return type:

list of str

Raises:

RuntimeError – When no site has been registered (_active_site is None) or the site has no file_list_fn.

monitoring.get_modal_results(quantity: str, duration: Timedelta, stats: Dataset = None, create_new: bool = False, **kwargs)

Load or (re-)create the modal-results database for quantity.

Parameters:

• quantity (str) – Quantity tag (e.g. 'accel', 'strain_rosettes').

• duration (pandas.Timedelta) – Analysis window length used to locate the database file.

• stats (xarray.Dataset or None, optional) – Statistics dataset required when create_new is True.

• create_new (bool, optional) – When True, run create_modal_results() instead of reading from disk. Defaults to False.

• **kwargs – Forwarded to create_modal_results() when create_new is True.

Returns:

Modal-results dataset with variables frequencies, damping, modeshapes, MPC, MPD, num_modes, model_orders, and associated per-channel PSD statistics, indexed by (time, modes, channels).

Return type:

xarray.Dataset

Raises:

RuntimeError – When create_new is True but stats is None.

monitoring.get_site(name: str) → Site

Retrieve a previously registered site by name.

Parameters:

name (str) – The site.name used when register_site was called.

Returns:

The registered Site instance.

Return type:

Site

Raises:

KeyError – When no site with name has been registered.

monitoring.get_slice(start_time, duration, quantity, file_info, upsample_fs=None)

channels ‘Tagessekunden’ and ‘Time’ are always dropped

monitoring.get_slice_corrected(start_time: Timestamp, duration: Timedelta, quantity: str, file_info: Dataset = None, **kwargs)

Return a site-transformed signal slice, loading from cache when possible.

First checks whether a pre-processed .npz slice already exists on disk (under slice_root_path/<duration>-minutes/slices_<quantity>/). If it does, the cached slice is returned directly. Otherwise, the raw slice is obtained via get_slice(), the site’s transforms callback is applied, and the result is saved for future calls.

Parameters:

• start_time (pandas.Timestamp) – Berlin-aware start of the requested window.

• duration (pandas.Timedelta) – Length of the requested window.

• quantity (str) – Quantity tag (e.g. 'accel', 'strain_rosettes').

• file_info (xarray.Dataset or None, optional) – File-info dataset required when the cache is empty and a raw slice must be assembled. May be None if a cached slice exists.

• **kwargs – Forwarded to the site’s transforms callback.

Returns:

(start_time, headers, units, end_time, sample_rate, measurement) or None when no data are available for the requested window.

Return type:

tuple or None

monitoring.get_slice_preprocessed(start_time: Timestamp, duration, quantity, file_info=None, **kwargs)

remove channels, detrend, filter bandpass and decimate

monitoring.get_stats(quantity: str, duration: Timedelta, file_info: Dataset = None, create_new: bool = False, **kwargs)

Load or (re-)create the statistics database for quantity.

Parameters:

• quantity (str) – Quantity tag (e.g. 'accel', 'wind').

• duration (pandas.Timedelta) – Analysis window length (e.g. pd.Timedelta('2h') for 120-minute statistics).

• file_info (xarray.Dataset or None, optional) – File-info dataset required when create_new is True.

• create_new (bool, optional) – When True, run create_stats() instead of reading from the existing NetCDF file. Defaults to False.

• **kwargs – Forwarded to create_stats() when create_new is True.

Returns:

Statistics dataset with variables mean, min, max, var, skewness, kurtosis, q05, q50, q95, rms, and error indexed by (time, channels).

Return type:

xarray.Dataset

Raises:

RuntimeError – When create_new is True but file_info is None.

monitoring.get_synchronized_time(start_time, file_time, duration)

Return the synchronised start time via the active site’s sync policy.

monitoring.modal_analysis_single(start_time, data_slice, quantity, duration)

Run VarSSI-Ref on a single preprocessed slice and return pole results.

Loads or computes three sequential artefacts (cached per window):

1. prep_data.npz — PreProcessSignals state including the singular-value PSD.

2. modal_data.npz — VarSSIRef state matrix.

3. stabil_data.npz — StabilCluster state with automatic clearing, classification, and pole selection.

A stabilization-diagram PNG is also written to the result folder.

Parameters:

• start_time (pandas.Timestamp) – Berlin-aware start of the analysis window (used to name result files and locate the configuration directory).

• data_slice (tuple) – (start_time, headers, units, end_time, sample_rate, measurement) as returned by get_slice_preprocessed().

• quantity (str) – Quantity tag (e.g. 'accel'); used to locate the modal_conf_dir/<quantity>/ configuration directory.

• duration (pandas.Timedelta) – Window length; determines the result subfolder name.

Returns:

[model_orders, frequencies, std_frequencies, damping, std_damping, MPC, MP, MPD, MC, modeshapes, s_vals_psd] as returned by return_results() plus the singular-value PSD matrix.

Return type:

list

Raises:

RuntimeError – When no setup_prep callback is registered for quantity.

monitoring.read_file(path)

read binary file from path return array, start_time, duration, headers, path may be a csv,dat or bin file or a bzip2ed version of it or an incomplete path refering to the 8 files generated by illumisense (common path part of these files, ends in underscore, has not extension)

illumisense files may be .txt or .txt.bz2

we first try to read a .npz version for this file, if it exists then we try to open the bz2 file and pass the file descriptor to the following function then we call the respective function to read the file

the illumisense reader must handle bz2 internally

Short comparison of slowdown and space saving using bz2. packed files

time speedup/slowdown [%] filesize space saving [%]

.dat 1,64E-07 100,00 14400768 .dat.bz2 1,27E-06 12,91 10300268 28,47

.csv 1,85E-07 100,00 40320184 .csv.bz2 9,55E-07 19,37 9115468 77,39

.txt 8,32E-07 100,00 3260560549 .txt.bz2 2,29E-06 36,30 217251268 93,34 .npz 3,92E-08 2125,37

.bin 1,26E-04 100,00 34195960 .bin.bz2 1,26E-04 100,00 2882224 91,57 .npz 4,90E-07 25617,82

monitoring.register_site(site: Site) → None

Add site to the module-level site registry under site.name.

Parameters:

site (Site) – A fully configured Site dataclass instance.

monitoring.round_dt(dt: datetime64, duration: timedelta64, ceil: bool = True, floor: bool = False)

Round dt up (or down) to the nearest multiple of duration.

Parameters:

• dt (numpy.datetime64) – The datetime to round.

• duration (numpy.timedelta64) – Rounding grid size (e.g. np.timedelta64(120, 'm') for 2-hour intervals).

• ceil (bool, optional) – Round toward the next grid point (default True).

• floor (bool, optional) – Round toward the previous grid point. Setting this to True forces ceil to False.

Returns:

dt rounded to the nearest duration boundary.

Return type:

numpy.datetime64

monitoring.save_ds(new_ds, current_ds, savepath, what='modal', reload_current=False)

defined behaviour:

when creating new results:

save to a unique netcdf file for each process in order to avoid loosing data unique netcdf files will be merged later

when updating results:

iterate over stats and existing modal results regenerate where necessary save to a unique netcdf file for each process

upon process exit

lock main netcdf merge main netcdf with processes netcdf while dropping conflicting indexes from main file

monitoring.set_active_site(site: Site) → None

Mark site as the process-level active site.

All engine functions that depend on site configuration (channel lists, path roots, callbacks) read _active_site instead of reaching into config.py directly.

Parameters:

site (Site) – A Site instance that has already been passed to register_site().

monitoring.split_modepairs(modal)

Reorder raw poles into named mode-pair slots defined by the active site.

For each frequency band in site.modal_bands, the function:

1. Selects time steps where exactly two poles fall inside the band.

2. Assigns the lower-frequency pole to slot 2*i and the higher-frequency pole to slot 2*i+1 (where i is the band index).

3. Merges all per-band results into a single dataset with a new modes coordinate of size 2 * len(modal_bands).

All other dataset variables (damping, modeshapes, MPC, etc.) are reordered to match the new mode assignment.

Parameters:

modal (xarray.Dataset) – Raw modal-results dataset as returned by get_modal_results(), with a sparse modes dimension.

Returns:

Dataset with a dense modes coordinate of length 2 * len(site.modal_bands), aligned across all time steps.

Return type:

xarray.Dataset

time_convention — timezone policy

Single source of truth for timezone/format conversions in the monitoring pipeline.

All methods are vectorised: they accept and return arrays / Series / xarray objects. No per-scalar wrappers are created.

class time_convention.TimeConvention(tz: str = 'Europe/Berlin')

Bases: object

Centralised timezone policy and conversion helpers.

Owns the project timezone and the canonical on-disk storage format.

The storage format uses two encodings for the same instant:

• time coordinate — tz-naive UTC datetime64[ns] (xarray index).

• start_time / file_time variables — float64 POSIX seconds.

tz

The project timezone, e.g. pytz.timezone('Europe/Berlin').

Type:

pytz.BaseTzInfo

gap_lengths(start_times, durations, sample_rates) → ndarray

Compute gap lengths in samples between consecutive files.

Calculates, for each file i, how many samples are missing between the end of file i and the start of file *i*+1. The last element is always np.nan because there is no next file.

Two correctness issues from the original implementation are fixed here:

• Bug 1 (unit) — the duration column (float seconds) is multiplied by np.timedelta64(1, 's') rather than 'us', so 3600 s adds one hour, not 3.6 ms.

• Bug 2 (NaT sentinel) — .shift(time=-1) fills the last element with NaT; casting NaT directly to int64 yields INT64_MIN (≈ −9.2×10¹⁸), which would propagate as an enormous negative gap. Elements equal to INT64_MIN are masked to np.nan before scaling.

Parameters:

• start_times (xarray.DataArray) – Float64 POSIX start times in seconds (the start_time variable from the file-info database).

• durations (xarray.DataArray) – Float64 recording durations in seconds.

• sample_rates (xarray.DataArray) – Float64 sample rates in Hz.

Returns:

Gap lengths in samples. The last element is np.nan. Negative values indicate file overlap; zero means perfectly consecutive.

Return type:

numpy.ndarray

is_near_dst_transition(t, hours: int = 3) → bool

Return True if t is within ±*hours* of any DST transition.

Used in monitoring.close_to_utc_transition to skip files recorded during the ±3-hour window around clock changes, where device timestamps are unreliable.

The comparison strips tzinfo from t before comparing against the naive UTC transition times stored in pytz, matching the original behaviour of the inline check this method replaces.

Parameters:

• t (datetime.datetime) – A datetime object (timezone-aware or timezone-naive).

• hours (int, optional) – Half-width of the exclusion window in hours. Defaults to 3.

Returns:

True if t falls within hours of any DST transition in the project timezone; False otherwise.

Return type:

bool

make_index(dtstart, until, minutes: int)

Build the regular time index used by create_stats and create_modal_results.

Generates a dateutil.rrule sequence with step minutes from dtstart to until (inclusive) and returns it in two parallel forms: a list of Berlin-aware pd.Timestamp objects (for local-time display and slice naming) and a NumPy array of tz-naive UTC datetime64[ns] values (for xarray database lookup).

Parameters:

• dtstart (datetime-like) – Start of the iteration range (Berlin-naive or Berlin-aware).

• until (datetime-like) – End of the iteration range, inclusive (same timezone convention as dtstart).

• minutes (int) – Step size in minutes (e.g. 30, 60, 120).

Returns:

• aware_list (list of pd.Timestamp) – Berlin-aware timestamps at every step from dtstart to until.

• naive_array (numpy.ndarray of datetime64[ns]) – Corresponding tz-naive UTC datetime64[ns] values, suitable for set-difference operations against the database time coordinate.

posix_to_datetime64(posix_s)

Convert float POSIX seconds to datetime64[s] (vectorised).

Convenience wrapper around .astype('datetime64[s]') that accepts both plain NumPy arrays and xarray DataArray objects.

Parameters:

posix_s (float, array-like, or xarray.DataArray) – POSIX timestamp(s) in seconds since the Unix epoch (1970-01-01 UTC).

Returns:

datetime64[s] representation of the same instant(s).

Return type:

numpy.ndarray or xarray.DataArray

to_local(stored) → Timestamp

Convert a stored tz-naive coordinate back to a Berlin-aware Timestamp.

Used in create_stats and create_modal_results to recover a Berlin-aware pd.Timestamp from the UTC-naive value that is stored in the database.

Note

This applies the “double-localisation” pattern: the stored UTC-naive value is re-localised as Berlin (not converted from UTC). Times that fall inside the spring-forward DST gap (02:00–02:59 on the last Sunday in March in Europe/Berlin) map to NaT rather than raising, because nonexistent='NaT' is passed to tz_localize.

Parameters:

stored (numpy.datetime64, pd.Timestamp, or str) – A tz-naive datetime value as stored in the database time coordinate.

Returns:

Berlin-aware timestamp, or NaT if stored falls inside a DST spring-forward gap.

Return type:

pd.Timestamp

to_posix(dt) → float

Convert a timezone-aware datetime to float POSIX seconds.

Parameters:

dt (datetime.datetime or pd.Timestamp) – A timezone-aware datetime.

Returns:

Seconds since the Unix epoch (1970-01-01 00:00:00 UTC), including fractional seconds.

Return type:

float

to_storage_coord(aware) → datetime64

Convert a timezone-aware datetime to a tz-naive UTC datetime64.

This is the canonical conversion used before writing a time coordinate to the NetCDF database. It is equivalent to:

pd.Timestamp(aware).tz_convert('UTC').tz_localize(None).to_datetime64()

Parameters:

aware (datetime.datetime or pd.Timestamp) – A timezone-aware datetime. pytz-aware and dateutil-aware objects are both accepted.

Returns:

Tz-naive UTC datetime64 suitable for use as an xarray time coordinate.

Return type:

numpy.datetime64

config — YAML loader

YAML-based configuration loader for the tower monitoring pipeline.

All static site configuration lives in config.yaml next to this file. This module loads that file, validates its structure, and re-exports every value as a module-level attribute so that existing code using import config; config.origins continues to work without modification.

Runtime-only state (file_cache, ds_cache, pid) is also initialised here and is not part of the YAML file.

config.load_config(path: str = '/home/docs/checkouts/readthedocs.org/user_builds/pyoma-monitoring/checkouts/latest/config.yaml') → dict

Load path as YAML and return the validated configuration dict.

Parameters:

path – Path to a YAML file with the structure defined in config.yaml.

Raises:

• FileNotFoundError – When path does not exist.

• yaml.YAMLError – When the file is not valid YAML.

• ValueError – When required keys are missing or values have wrong types.

config.validate_config(cfg: dict) → None

Validate the structure of a loaded configuration dict.

Raises ValueError with a descriptive message on the first violation found.

Parameters:

cfg – Dict produced by load_config() (or yaml.safe_load).

MultiLock — concurrent file access

File-based advisory lock for safe concurrent NetCDF database access.

Wraps simpleflock with an additional per-process lock file layer so that multiple worker processes can safely read and write the same NetCDF database without data corruption.

class MultiLock.MultiLock(path)

Bases: object

Context manager that provides exclusive access to a file path.

Layers a per-process .pid.lock sentinel file on top of simpleflock to close the race window where simpleflock can briefly grant the system lock to two processes simultaneously. The calling process:

1. Acquires the simpleflock system lock on <path>.lock.

2. Creates its own <path>.<pid>.lock sentinel file.

3. Releases the system lock.

4. Loops until no other sentinel files exist, sleeping a random sub-second interval on each iteration to break ties.

This guarantees that at most one process holds the advisory lock at any time, even when the underlying simpleflock implementation has a brief grant-to-two bug.

Parameters:

path (str or os.PathLike) – Absolute path to the file being protected (typically a NetCDF database such as modal_accel.nc).

Examples

with MultiLock('/path/to/database.nc'):
    ds = xr.open_dataset('/path/to/database.nc')
    # modify ds …
    ds.to_netcdf('/path/to/database.nc')

site_example — site template

Example / template site module for pyOMA-Monitoring.

Copy this file, rename it site_<yoursite>.py, and fill in every raise NotImplementedError stub with your site-specific logic. Import the finished module once (e.g. at the top of daily.py) to register and activate your site in the monitoring engine:

import site_mysite  # registers on import

All engine functions in monitoring.py will then use your callbacks automatically.

site_example.EXAMPLE_ALL_CHANNELS: Dict[str, List[str]] = {'accel': ['Accel_01', 'Accel_02']}

Map quantity → list of channel names that must be present in every slice.

site_example.EXAMPLE_DB_ROOT_PATH: str = '/path/to/results/db/'

Absolute path to the root directory that holds the result databases (file_info_*.nc, stats_*.nc, modal_*.nc).

site_example.EXAMPLE_DTSTARTS: Dict[str, object] = {'accel': '2020-01-01'}

Map origin tag → earliest available datetime (ISO string or datetime.datetime). Slices before this timestamp are skipped.

site_example.EXAMPLE_ERROR_RULES: Dict[str, dict] = {'accel': {'kurtosis_max': 5, 'kurtosis_min': -2}}

Map quantity → dict of kurtosis thresholds used by describe_stats. Slices whose kurtosis falls outside [kurtosis_min, kurtosis_max] are flagged as erroneous. Omit a quantity to apply no kurtosis check.

site_example.EXAMPLE_FILE_ROOT_PATH: str = '/path/to/raw_data/'

Absolute path to the root directory that holds the raw measurement files.

site_example.EXAMPLE_MODAL_BANDS: List[Tuple[float, float]] = [(0.5, 1.0)]

List of (f_lo, f_hi) Hz bands passed to split_modepairs when assigning poles to named modes. Add one tuple per expected mode.

site_example.EXAMPLE_MODAL_CONF_DIR: str = '/path/to/modal_conf/'

Absolute path to the directory that contains the OMA configuration files (nodes, lines, master_slaves, ssi_config, …).

site_example.EXAMPLE_OPTIONAL_CHANNELS: Dict[str, List[str]] = {'accel': []}

Map quantity → list of channel names that may be present (not required).

site_example.EXAMPLE_ORIGINS: Dict[str, str] = {'accel': 'accel'}

Map quantity → origin tag. The origin tag is the key used in subpaths, all_channels, and dtstarts. Add one entry per measured quantity.

Example:

EXAMPLE_ORIGINS = {
    "accel": "accel",
    "wind":  "wind",
}

site_example.EXAMPLE_PREPROC_CHANNELS: Dict[str, List[str]] = {'accel': ['Accel_01', 'Accel_02']}

Map quantity → list of channel names kept after the band-pass / decimation pre-processing step. Only relevant for quantities that go through OMA (typically "accel" and strain-type quantities).

site_example.EXAMPLE_RANGES: Dict[str, Tuple[float, float]] = {'Accel_01': (-20.0, 20.0)}

Map channel name → (min, max) plausibility range. Samples outside this interval cause the affected time slice to be flagged as erroneous.

site_example.EXAMPLE_SLICE_ROOT_PATH: str = '/path/to/slices/'

Absolute path to the root directory used for pre-processed signal slices.

site_example.EXAMPLE_SUBPATHS: Dict[str, str] = {'accel': 'accel/'}

Map origin tag → relative sub-path under EXAMPLE_FILE_ROOT_PATH.

site_example.register_example_site() → None

Register and activate the example site in the monitoring engine.

Called automatically when this module is imported. Safe to call multiple times — it simply overwrites the registry entry for "example".

Raises:

NotImplementedError – Any stub callback that has not been implemented yet will raise this error at runtime when the engine first invokes it.