API Documentation

Connectors

Base

class pastastore.base.BaseConnector[source]

Base Connector class.

Class holds base logic for dealing with time series and Pastas Models. Create your own Connector to a data source by writing a a class that inherits from this BaseConnector. Your class has to override each abstractmethod and abstractproperty.

CHECK_MODEL_SERIES_VALUES = True

USE_PASTAS_VALIDATE_SERIES = True

abstract _add_item(libname: str, item: DataFrame | Series | Dict, name: str, metadata: Dict | None = None, overwrite: bool = False) → None[source]

Internal method to add item for both time series and pastas.Models.

Must be overriden by subclass.

Parameters:

libname (str) – name of library to add item to
item (FrameorSeriesUnion or dict) – item to add
name (str) – name of the item
metadata (dict, optional) – dictionary containing metadata, by default None

_add_oseries_model_links(onam: str, mlnames: str | List[str])[source]

Add model name to stored list of models per oseries.

Parameters:

onam (str) – name of oseries
mlnames (Union[str, List[str]]) – model name or list of model names for an oseries with name onam.

_add_series(libname: str, series: DataFrame | Series, name: str, metadata: dict | None = None, validate: bool | None = None, overwrite: bool = False) → None[source]

Internal method to add series to database.

Parameters:

libname (str) – name of the library to add the series to
series (pandas.Series or pandas.DataFrame) – data to add
name (str) – name of the time series
metadata (dict, optional) – dictionary containing metadata, by default None
validate (bool, optional) – use pastas to validate series, default is None, which will use the USE_PASTAS_VALIDATE_SERIES value (default is True).
overwrite (bool, optional) – overwrite existing dataset with the same name, by default False

Raises:

ItemInLibraryException – if overwrite is False and name is already in the database

static _clear_cache(libname: str) → None[source]: Clear cached property.

_default_library_names = ['oseries', 'stresses', 'models', 'oseries_models']

abstract _del_item(libname: str, name: str) → None[source]

Internal method to delete items (series or models).

Must be overriden by subclass.

Parameters:

libname (str) – name of library to delete item from
name (str) – name of item to delete

_del_oseries_model_link(onam, mlnam)[source]

Delete model name from stored list of models per oseries.

Parameters:

onam (str) – name of oseries
mlnam (str) – name of model

_get_all_oseries_model_links()[source]

Get all model names per oseries in dictionary.

Returns:: links – dictionary with oseries names as keys and lists of model names as values
Return type:: dict

abstract _get_item(libname: str, name: str) → DataFrame | Series | Dict[source]

Internal method to get item (series or pastas.Models).

Must be overriden by subclass.

Parameters:

libname (str) – name of library
name (str) – name of item

Returns:

item – item (time series or pastas.Model)

Return type:

FrameorSeriesUnion or dict

abstract _get_library(libname: str)[source]

Get library handle.

Must be overriden by subclass.

Parameters:: libname (str) – name of the library
Returns:: lib – handle to the library
Return type:: Any

abstract _get_metadata(libname: str, name: str) → Dict[source]

Internal method to get metadata.

Must be overriden by subclass.

Parameters:

libname (str) – name of the library
name (str) – name of the item

Returns:

metadata – dictionary containing metadata

Return type:

dict

_get_series(libname: str, names: list | str, progressbar: bool = True, squeeze: bool = True) → DataFrame | Series[source]

Internal method to get time series.

Parameters:

libname (str) – name of the library
names (str or list of str) – names of the time series to load
progressbar (bool, optional) – show progressbar, by default True
squeeze (bool, optional) – if True return DataFrame or Series instead of dictionary for single entry

Returns:

either returns time series as pandas.DataFrame or dictionary containing the time series.

Return type:

pandas.DataFrame or dict of pandas.DataFrames

_iter_series(libname: str, names: List[str] | None = None)[source]

Internal method iterate over time series in library.

Parameters:

libname (str) – name of library (e.g. ‘oseries’ or ‘stresses’)
names (Optional[List[str]], optional) – list of names, by default None, which defaults to all stored series

Yields:

pandas.Series or pandas.DataFrame – time series contained in library

property _modelnames_cache: List of model names.

static _parse_series_input(series: DataFrame | Series, metadata: Dict | None = None) → Tuple[DataFrame | Series, Dict | None][source]

Internal method to parse series input.

Parameters:

series (FrameorSeriesUnion,) – series object to parse
metadata (dict, optional) – metadata dictionary or None, by default None

Returns:

series, metadata – time series as pandas.Series or DataFrame and optionally metadata dictionary

Return type:

FrameorSeriesUnion, Optional[Dict]

_pastas_validate(validate)[source]

Whether to validate time series.

Parameters:: validate (bool, NoneType) – value of validate keyword argument
Returns:: b – return global or local setting (True or False)
Return type:: bool

_update_all_oseries_model_links()[source]

Add all model names to oseries metadata dictionaries.

Used for old PastaStore versions, where relationship between oseries and models was not stored. If there are any models in the database and if the oseries_models library is empty, loops through all models to determine which oseries each model belongs to.

_update_series(libname: str, series: DataFrame | Series, name: str, metadata: dict | None = None, validate: bool | None = None) → None[source]

Internal method to update time series.

Parameters:

libname (str) – name of library
series (FrameorSeriesUnion) – time series containing update values
name (str) – name of the time series to update
metadata (Optional[dict], optional) – optionally provide metadata dictionary which will also update the current stored metadata dictionary, by default None
validate (bool, optional) – use pastas to validate series, default is None, which will use the USE_PASTAS_VALIDATE_SERIES value (default is True).

_upsert_series(libname: str, series: DataFrame | Series, name: str, metadata: dict | None = None, validate: bool | None = None) → None[source]

Update or insert series depending on whether it exists in store.

Parameters:

libname (str) – name of library
series (FrameorSeriesUnion) – time series to update/insert
name (str) – name of the time series
metadata (Optional[dict], optional) – metadata dictionary, by default None
validate (bool, optional) – use pastas to validate series, default is None, which will use the USE_PASTAS_VALIDATE_SERIES value (default is True).

add_model(ml: Model | dict, overwrite: bool = False, validate_metadata: bool = False) → None[source]

Add model to the database.

Parameters:

ml (pastas.Model or dict) – pastas Model or dictionary to add to the database
overwrite (bool, optional) – if True, overwrite existing model, by default False
validate_metadata – remove unsupported characters from metadata dictionary keys
optional (bool) – remove unsupported characters from metadata dictionary keys

Raises:

TypeError – if model is not pastas.Model or dict
ItemInLibraryException – if overwrite is False and model is already in the database

add_oseries(series: DataFrame | Series, name: str, metadata: dict | None = None, validate: bool | None = None, overwrite: bool = False) → None[source]

Add oseries to the database.

Parameters:

series (pandas.Series or pandas.DataFrame) – data to add
name (str) – name of the time series
metadata (dict, optional) – dictionary containing metadata, by default None.
validate (bool, optional) – use pastas to validate series, default is None, which will use the USE_PASTAS_VALIDATE_SERIES value (default is True).
overwrite (bool, optional) – overwrite existing dataset with the same name, by default False

add_stress(series: DataFrame | Series, name: str, kind: str, metadata: dict | None = None, validate: bool | None = None, overwrite: bool = False) → None[source]

Add stress to the database.

Parameters:

series (pandas.Series or pandas.DataFrame) – data to add, if pastas.Timeseries is passed, series_orignal and metadata is stored in database
name (str) – name of the time series
kind (str) – category to identify type of stress, this label is added to the metadata dictionary.
metadata (dict, optional) – dictionary containing metadata, by default None.
validate (bool, optional) – use pastas to validate series, default is True
overwrite (bool, optional) – overwrite existing dataset with the same name, by default False

del_models(names: list | str) → None[source]

Delete model(s) from the database.

Parameters:: names (str or list of str) – name(s) of the model to delete

del_oseries(names: list | str, remove_models: bool = False)[source]

Delete oseries from the database.

Parameters:

names (str or list of str) – name(s) of the oseries to delete
remove_models (bool, optional) – also delete models for deleted oseries, default is False

del_stress(names: list | str)[source]

Delete stress from the database.

Parameters:: names (str or list of str) – name(s) of the stress to delete

empty_library(libname: str, prompt: bool = True, progressbar: bool = True)[source]

Empty library of all its contents.

Parameters:

libname (str) – name of the library
prompt (bool, optional) – prompt user for input before deleting contents, by default True. Default answer is “n”, user must enter ‘y’ to delete contents
progressbar (bool, optional) – show progressbar, by default True

get_metadata(libname: str, names: list | str, progressbar: bool = False, as_frame: bool = True, squeeze: bool = True) → dict | DataFrame[source]

Read metadata from database.

Parameters:

libname (str) – name of the library containing the dataset
names (str or list of str) – names of the datasets for which to read the metadata
squeeze (bool, optional) – if True return dict instead of list of dict for single entry

Returns:

returns metadata dictionary or DataFrame of metadata

Return type:

dict or pandas.DataFrame

get_models(names: list | str, return_dict: bool = False, progressbar: bool = False, squeeze: bool = True, update_ts_settings: bool = False) → Model | list[source]

Load models from database.

Parameters:

names (str or list of str) – names of the models to load
return_dict (bool, optional) – return model dictionary instead of pastas.Model (much faster for obtaining parameters, for example)
progressbar (bool, optional) – show progressbar, by default False
squeeze (bool, optional) – if True return Model instead of list of Models for single entry
update_ts_settings (bool, optional) – update time series settings based on time series in store. overwrites stored tmin/tmax in model.

Returns:

return pastas model, or list of models if multiple names were passed

Return type:

pastas.Model or list of pastas.Model

Get oseries from database.

Parameters:

names (str or list of str) – names of the oseries to load
return_metadata (bool, optional) – return metadata as dictionary or list of dictionaries, default is False
progressbar (bool, optional) – show progressbar, by default False
squeeze (bool, optional) – if True return DataFrame or Series instead of dictionary for single entry

Returns:

oseries (pandas.DataFrame or dict of DataFrames) – returns time series as DataFrame or dictionary of DataFrames if multiple names were passed
metadata (dict or list of dict) – metadata for each oseries, only returned if return_metadata=True

Get stresses from database.

Parameters:

names (str or list of str) – names of the stresses to load
return_metadata (bool, optional) – return metadata as dictionary or list of dictionaries, default is False
progressbar (bool, optional) – show progressbar, by default False
squeeze (bool, optional) – if True return DataFrame or Series instead of dictionary for single entry

Returns:

stresses (pandas.DataFrame or dict of DataFrames) – returns time series as DataFrame or dictionary of DataFrames if multiple names were passed
metadata (dict or list of dict) – metadata for each stress, only returned if return_metadata=True

iter_models(modelnames: List[str] | None = None, return_dict: bool = False)[source]

Iterate over models in library.

Parameters:

modelnames (Optional[List[str]], optional) – list of models to iterate over, by default None which uses all models
return_dict (bool, optional) – if True, return model as dictionary, by default False, which returns a pastas.Model.

Yields:

pastas.Model or dict – time series model

iter_oseries(names: List[str] | None = None)[source]

Iterate over oseries in library.

Parameters:: names (Optional[List[str]], optional) – list of oseries names, by default None, which defaults to all stored series
Yields:: pandas.Series or pandas.DataFrame – oseries contained in library

iter_stresses(names: List[str] | None = None)[source]

Iterate over stresses in library.

Parameters:: names (Optional[List[str]], optional) – list of stresses names, by default None, which defaults to all stored series
Yields:: pandas.Series or pandas.DataFrame – stresses contained in library

abstract property model_names

List of model names.

Property must be overriden by subclass.

property n_models

property n_oseries

property n_stresses

property oseries: Dataframe with overview of oseries.

property oseries_models

List of model names per oseries.

Returns:: d – dictionary with oseries names as keys and list of model names as values
Return type:: dict

abstract property oseries_names

List of oseries names.

Property must be overriden by subclass.

set_check_model_series_values(b: bool)[source]

Turn CHECK_MODEL_SERIES_VALUES option on (True) or off (False).

The default option is on (it is highly recommended to keep it that way). When turned on, the model time series (ml.oseries._series_original, and stressmodel.stress._series_original) values are checked against the stored copies in the database. If these do not match, an error is raised, and the model is not added to the database. This guarantees the stored model will be identical after loading from the database. This check is somewhat computationally expensive, which is why it can be turned on or off.

Parameters:: b (bool) – boolean indicating whether option should be turned on (True) or off (False). Option is on by default.

set_use_pastas_validate_series(b: bool)[source]

Turn USE_PASTAS_VALIDATE_SERIES option on (True) or off (False).

This will use pastas.validate_oseries() or pastas.validate_stresses() to test the time series. If they do not meet the criteria, an error is raised. Turning this option off will allow the user to store any time series but this will mean that time series models cannot be made from stored time series directly and will have to be modified before building the models. This in turn will mean that storing the models will not work as the stored time series copy is checked against the time series in the model to check if they are equal.

Note: this option requires pastas>=0.23.0, otherwise it is turned off.

Parameters:: b (bool) – boolean indicating whether option should be turned on (True) or off (False). Option is on by default.

property stresses: Dataframe with overview of stresses.

abstract property stresses_names

List of stresses names.

Property must be overriden by subclass.

update_metadata(libname: str, name: str, metadata: dict) → None[source]

Update metadata.

Note: also retrieves and stores time series as updating only metadata is not supported for some Connectors.

Parameters:

libname (str) – name of library
name (str) – name of the item for which to update metadata
metadata (dict) – metadata dictionary that will be used to update the stored metadata

update_oseries(series: DataFrame | Series, name: str, metadata: dict | None = None) → None[source]

Update oseries values.

Parameters:

series (FrameorSeriesUnion) – time series to update stored oseries with
name (str) – name of the oseries to update
metadata (Optional[dict], optional) – optionally provide metadata, which will update the stored metadata dictionary, by default None

update_stress(series: DataFrame | Series, name: str, metadata: dict | None = None) → None[source]

Update stresses values.

Note: the ‘kind’ attribute of a stress cannot be updated! To update the ‘kind’ delete and add the stress again.

Parameters:

series (FrameorSeriesUnion) – time series to update stored stress with
name (str) – name of the stress to update
metadata (Optional[dict], optional) – optionally provide metadata, which will update the stored metadata dictionary, by default None

upsert_oseries(series: DataFrame | Series, name: str, metadata: dict | None = None) → None[source]

Update or insert oseries values depending on whether it exists.

Parameters:

series (FrameorSeriesUnion) – time series to update/insert
name (str) – name of the oseries
metadata (Optional[dict], optional) – optionally provide metadata, which will update the stored metadata dictionary if it exists, by default None

upsert_stress(series: DataFrame | Series, name: str, kind: str, metadata: dict | None = None) → None[source]

Update or insert stress values depending on whether it exists.

Parameters:

series (FrameorSeriesUnion) – time series to update/insert
name (str) – name of the stress
metadata (Optional[dict], optional) – optionally provide metadata, which will update the stored metadata dictionary if it exists, by default None

class pastastore.base.ConnectorUtil[source]

Mix-in class for general Connector helper functions.

Only for internal methods, and not methods that are related to CRUD operations on database.

static _check_model_series_names_for_store(ml)[source]

_check_oseries_in_store(ml: Model | dict)[source]

Internal method, check if Model oseries are contained in PastaStore.

Parameters:: ml (Union[ps.Model, dict]) – pastas Model

_check_stresses_in_store(ml: Model | dict)[source]

Internal method, check if stresses time series are contained in PastaStore.

Parameters:: ml (Union[ps.Model, dict]) – pastas Model

static _check_stressmodels_supported(ml)[source]

_get_model_orphans()[source]

Get models whose oseries no longer exist in database.

Returns:: dictionary with oseries names as keys and lists of model names as values
Return type:: dict

static _meta_list_to_frame(metalist: list, names: list)[source]

Convert list of metadata dictionaries to DataFrame.

Parameters:

metalist (list) – list of metadata dictionaries
names (list) – list of names corresponding to data in metalist

Returns:

DataFrame containing overview of metadata

Return type:

pandas.DataFrame

static _metadata_from_json(fjson: str)[source]

Load metadata dictionary from JSON.

Parameters:: fjson (str) – path to file
Returns:: meta – dictionary containing metadata
Return type:: dict

_models_to_archive(archive, names=None, progressbar=True)[source]

Internal method for writing pastas.Model to zipfile.

Parameters:

archive (zipfile.ZipFile) – reference to an archive to write data to
names (str or list of str, optional) – names of the models to write to archive, by default None, which writes all models to archive
progressbar (bool, optional) – show progressbar, by default True

_parse_model_dict(mdict: dict, update_ts_settings: bool = False)[source]

Internal method to parse dictionary describing pastas models.

Parameters:

mdict (dict) – dictionary describing pastas.Model
update_ts_settings (bool, optional) – update stored tmin and tmax in time series settings based on time series loaded from store.

Returns:

ml – time series analysis model

Return type:

pastas.Model

_parse_names(names: list | str | None = None, libname: str | None = 'oseries') → list[source]

Internal method to parse names kwarg, returns iterable with name(s).

Parameters:

names (Union[list, str], optional) – str or list of str or None or ‘all’ (last two options retrieves all names)
libname (str, optional) – name of library, default is ‘oseries’

Returns:

list of names

Return type:

list

static _series_from_json(fjson: str)[source]

Load time series from JSON.

Parameters:: fjson (str) – path to file
Returns:: s – DataFrame containing time series
Return type:: pd.DataFrame

_series_to_archive(archive, libname: str, names: list | str | None = None, progressbar: bool = True)[source]

Internal method for writing DataFrame or Series to zipfile.

Parameters:

archive (zipfile.ZipFile) – reference to an archive to write data to
libname (str) – name of the library to write to zipfile
names (str or list of str, optional) – names of the time series to write to archive, by default None, which writes all time series to archive
progressbar (bool, optional) – show progressbar, by default True

static _set_series_name(series, name)[source]

Set series name to match user defined name in store.

Parameters:

series (pandas.Series or pandas.DataFrame) – set name for this time series
name (str) – name of the time series (used in the pastastore)

_stored_metadata_to_json(libname: str, names: list | str | None = None, squeeze: bool = True, progressbar: bool = False)[source]

Write metadata from stored series to JSON.

Parameters:

libname (str) – library containing series
names (Optional[Union[list, str]], optional) – names to parse, by default None
squeeze (bool, optional) – return single entry as json string instead of list, by default True
progressbar (bool, optional) – show progressbar, by default False

Returns:

files – list of json string

Return type:

list or str

_stored_series_to_json(libname: str, names: list | str | None = None, squeeze: bool = True, progressbar: bool = False)[source]

Write stored series to JSON.

Parameters:

libname (str) – library name
names (Optional[Union[list, str]], optional) – names of series, by default None
squeeze (bool, optional) – return single entry as json string instead of list, by default True
progressbar (bool, optional) – show progressbar, by default False

Returns:

files – list of series converted to JSON string or single string if single entry is returned and squeeze is True

Return type:

list or str

static _validate_input_series(series)[source]

check if series is pandas.DataFrame or pandas.Series.

Parameters:: series (object) – object to validate
Raises:: TypeError – if object is not of type pandas.DataFrame or pandas.Series

class pastastore.base.ModelAccessor(conn)[source]

Object for managing access to stored models.

Provides dict-like access to models (i.e. PastaStore.models[“model1”]), or allows adding models to the PastaStore using dict-like assignment (i.e. PastaStore.models[“model1”] = ml), and it can serve as an iterator (i.e. [ml for ml in pstore.models]).

random()[source]

DictConnector

class pastastore.DictConnector(name: str = 'pastas_db')[source]

Bases: BaseConnector, ConnectorUtil

_add_item(libname: str, item: DataFrame | Series | Dict, name: str, metadata: Dict | None = None, **_) → None[source]

Internal method to add item (time series or models).

Parameters:

libname (str) – name of library
item (FrameorSeriesUnion) – pandas.Series or pandas.DataFrame containing data
name (str) – name of the item
metadata (dict, optional) – dictionary containing metadata, by default None

_del_item(libname: str, name: str) → None[source]

Internal method to delete items (series or models).

Parameters:

libname (str) – name of library to delete item from
name (str) – name of item to delete

_get_item(libname: str, name: str) → DataFrame | Series | Dict[source]

Internal method to retrieve item from pystore library.

Parameters:

libname (str) – name of the library
name (str) – name of the item

Returns:

item – time series or model dictionary

Return type:

Union[FrameorSeriesUnion, Dict]

_get_library(libname: str)[source]

Get reference to dictionary holding data.

Parameters:: libname (str) – name of the library
Returns:: lib – library handle
Return type:: dict

_get_metadata(libname: str, name: str) → dict[source]

Internal method to read metadata.

Parameters:

libname (str) – name of the library the series are in (“oseries” or “stresses”)
name (str) – name of item to load metadata for

Returns:

imeta – dictionary containing metadata

Return type:

dict

property model_names: List of model names.

property oseries_names: List of oseries names.

property oseries_with_models: List of oseries with models.

property stresses_names: List of stresses names.

PasConnector

class pastastore.PasConnector(name: str, path: str)[source]

Bases: BaseConnector, ConnectorUtil

_add_item(libname: str, item: DataFrame | Series | Dict, name: str, metadata: Dict | None = None, **_) → None[source]

Internal method to add item (time series or models).

Parameters:

libname (str) – name of library
item (FrameorSeriesUnion) – pandas.Series or pandas.DataFrame containing data
name (str) – name of the item
metadata (dict, optional) – dictionary containing metadata, by default None

_del_item(libname: str, name: str) → None[source]

Internal method to delete items (series or models).

Parameters:

libname (str) – name of library to delete item from
name (str) – name of item to delete

_get_item(libname: str, name: str) → DataFrame | Series | Dict[source]

Internal method to retrieve item.

Parameters:

libname (str) – name of the library
name (str) – name of the item

Returns:

item – time series or model dictionary

Return type:

Union[FrameorSeriesUnion, Dict]

_get_library(libname: str)[source]

Get path to directory holding data.

Parameters:: libname (str) – name of the library
Returns:: lib – path to library
Return type:: str

_get_metadata(libname: str, name: str) → dict[source]

Internal method to read metadata.

Parameters:

libname (str) – name of the library the series are in (“oseries” or “stresses”)
name (str) – name of item to load metadata for

Returns:

imeta – dictionary containing metadata

Return type:

dict

_initialize() → None[source]: Internal method to initialize the libraries.

property model_names: List of model names.

property oseries_names: List of oseries names.

property oseries_with_models: List of oseries with models.

property stresses_names: List of stresses names.

ArcticDBConnector

class pastastore.ArcticDBConnector(name: str, uri: str)[source]

Bases: BaseConnector, ConnectorUtil

_abc_impl = <_abc._abc_data object>

_add_item(libname: str, item: DataFrame | Series | Dict, name: str, metadata: Dict | None = None, **_) → None[source]

Internal method to add item to library (time series or model).

Parameters:

libname (str) – name of the library
item (Union[FrameorSeriesUnion, Dict]) – item to add, either time series or pastas.Model as dictionary
name (str) – name of the item
metadata (Optional[Dict], optional) – dictionary containing metadata, by default None

_del_item(libname: str, name: str) → None[source]

Internal method to delete items (series or models).

Parameters:

libname (str) – name of library to delete item from
name (str) – name of item to delete

_get_item(libname: str, name: str) → DataFrame | Series | Dict[source]

Internal method to retrieve item from library.

Parameters:

libname (str) – name of the library
name (str) – name of the item

Returns:

item – time series or model dictionary

Return type:

Union[FrameorSeriesUnion, Dict]

_get_library(libname: str)[source]

Get ArcticDB library handle.

Parameters:: libname (str) – name of the library
Returns:: lib – handle to the library
Return type:: arcticdb.Library handle

_get_metadata(libname: str, name: str) → dict[source]

Internal method to retrieve metadata for an item.

Parameters:

libname (str) – name of the library
name (str) – name of the item

Returns:

dictionary containing metadata

Return type:

dict

_initialize() → None[source]: Internal method to initalize the libraries.

_library_name(libname: str) → str[source]: Internal method to get full library name according to ArcticDB.

conn_type = 'arcticdb'

property model_names

List of model names.

Returns:: list of models in library
Return type:: list

property oseries_names

List of oseries names.

Returns:: list of oseries in library
Return type:: list

property oseries_with_models: List of oseries with models.

property stresses_names

List of stresses names.

Returns:: list of stresses in library
Return type:: list

ArcticConnector

class pastastore.ArcticConnector(name: str, connstr: str)[source]

Bases: BaseConnector, ConnectorUtil

_abc_impl = <_abc._abc_data object>

_add_item(libname: str, item: DataFrame | Series | Dict, name: str, metadata: Dict | None = None, **_) → None[source]

Internal method to add item to library (time series or model).

Parameters:

libname (str) – name of the library
item (Union[FrameorSeriesUnion, Dict]) – item to add, either time series or pastas.Model as dictionary
name (str) – name of the item
metadata (Optional[Dict], optional) – dictionary containing metadata, by default None

_del_item(libname: str, name: str) → None[source]

Internal method to delete items (series or models).

Parameters:

libname (str) – name of library to delete item from
name (str) – name of item to delete

_get_item(libname: str, name: str) → DataFrame | Series | Dict[source]

Internal method to retrieve item from library.

Parameters:

libname (str) – name of the library
name (str) – name of the item

Returns:

item – time series or model dictionary

Return type:

Union[FrameorSeriesUnion, Dict]

_get_library(libname: str)[source]

Get Arctic library handle.

Parameters:: libname (str) – name of the library
Returns:: lib – handle to the library
Return type:: arctic.Library handle

_get_metadata(libname: str, name: str) → dict[source]

Internal method to retrieve metadata for an item.

Parameters:

libname (str) – name of the library
name (str) – name of the item

Returns:

dictionary containing metadata

Return type:

dict

_initialize() → None[source]: Internal method to initalize the libraries.

_library_name(libname: str) → str[source]: Internal method to get full library name according to Arctic.

conn_type = 'arctic'

property model_names

List of model names.

Returns:: list of models in library
Return type:: list

property oseries_names

List of oseries names.

Returns:: list of oseries in library
Return type:: list

property oseries_with_models: List of oseries with models.

property stresses_names

List of stresses names.

Returns:: list of stresses in library
Return type:: list

PystoreConnector

class pastastore.PystoreConnector(name: str, path: str)[source]

Bases: BaseConnector, ConnectorUtil

_abc_impl = <_abc._abc_data object>

_add_item(libname: str, item: DataFrame | Series | Dict, name: str, metadata: Dict | None = None, overwrite: bool = False) → None[source]

Internal method to add item to library (time series or model).

Parameters:

libname (str) – name of the library
item (Union[FrameorSeriesUnion, Dict]) – item to add, either time series or pastas.Model as dictionary
name (str) – name of the item
metadata (Optional[Dict], optional) – dictionary containing metadata, by default None
overwrite (bool, optional) – overwrite item if it already exists, by default False.

_del_item(libname: str, name: str) → None[source]

Internal method to delete data from the store.

Parameters:

libname (str) – name of the library
name (str) – name of the item to delete

_get_item(libname: str, name: str) → DataFrame | Series | Dict[source]

Internal method to retrieve item from pystore library.

Parameters:

libname (str) – name of the library
name (str) – name of the item

Returns:

item – time series or model dictionary

Return type:

Union[FrameorSeriesUnion, Dict]

_get_library(libname: str)[source]

Get Pystore library handle.

Parameters:: libname (str) – name of the library
Returns:: handle to the library
Return type:: Pystore.Collection handle

_get_metadata(libname: str, name: str) → dict[source]

Internal method to read metadata from pystore.

Parameters:

libname (str) – name of the library the series are in (“oseries” or “stresses”)
name (str) – name of item to load metadata for

Returns:

imeta – dictionary containing metadata

Return type:

dict

_initialize() → None[source]: Internal method to initalize the libraries (stores).

conn_type = 'pystore'

property model_names

List of model names.

Returns:: list of models in library
Return type:: list

property oseries_names

List of oseries names.

Returns:: list of oseries in library
Return type:: list

property oseries_with_models: List of oseries with models.

property stresses_names

List of stresses names.

Returns:: list of stresses in library
Return type:: list

PastaStore

class pastastore.store.PastaStore(connector: BaseConnector | None = None, name: str | None = None)[source]

PastaStore object for managing pastas time series and models.

Requires a Connector object to provide the interface to the database. Different Connectors are available, e.g.:

PasConnector for storing all data as .pas (JSON) files on disk (recommended)
DictConenctor for storing all data in dictionaries (in-memory)
ArcticConnector for saving data to MongoDB using the Arctic module
PystoreConnector for saving data to disk using the Pystore module

Parameters:

connector (Connector object) – object that provides the interface to the database, e.g. ArcticConnector (see pastastore.connectors)
name (str, optional) – name of the PastaStore, by default takes the name of the Connector object

add_recharge(ml: Model, rfunc=None, recharge=None, recharge_name: str = 'recharge') → None[source]

Add recharge to a pastas model.

Uses closest precipitation and evaporation time series in database. These are assumed to be labeled with kind = ‘prec’ or ‘evap’.

Parameters:

ml (pastas.Model) – pastas.Model object
rfunc (pastas.rfunc, optional) – response function to use for recharge in model, by default None which uses ps.Exponential() (for different response functions, see pastas documentation)
recharge (ps.RechargeModel) – recharge model to use, default is ps.rch.Linear()
recharge_name (str) – name of the RechargeModel

apply(libname, func, names=None, progressbar=True)[source]

Apply function to items in library.

Supported libraries are oseries, stresses, and models.

Parameters:

libname (str) – library name, supports “oseries”, “stresses” and “models”
func (callable) – function that accepts items from one of the supported libraries as input
names (str, list of str, optional) – apply function to these names, by default None which loops over all stored items in library
progressbar (bool, optional) – show progressbar, by default True

Returns:

dict of results of func, with names as keys and results as values

Return type:

dict

create_model(name: str, modelname: str | None = None, add_recharge: bool = True, recharge_name: str = 'recharge') → Model[source]

Create a pastas Model.

Parameters:

name (str) – name of the oseries to create a model for
modelname (str, optional) – name of the model, default is None, which uses oseries name
add_recharge (bool, optional) – add recharge to the model by looking for the closest precipitation and evaporation time series in the stresses library, by default True
recharge_name (str) – name of the RechargeModel

Returns:

model for the oseries

Return type:

pastas.Model

Raises:

KeyError – if data is stored as dataframe and no column is provided
ValueError – if time series is empty

create_models_bulk(oseries: list | str | None = None, add_recharge: bool = True, solve: bool = False, store_models: bool = True, ignore_errors: bool = False, progressbar: bool = True, **kwargs) → Tuple[dict, dict] | dict[source]

Bulk creation of pastas models.

Parameters:

oseries (list of str, optional) – names of oseries to create models for, by default None, which creates models for all oseries
add_recharge (bool, optional) – add recharge to the models based on closest precipitation and evaporation time series, by default True
solve (bool, optional) – solve the model, by default False
store_models (bool, optional) – if False, return a list of models, by default True, which will store the models in the database.
ignore_errors (bool, optional) – ignore errors while creating models, by default False
progressbar (bool, optional) – show progressbar, by default True

Returns:

models (dict, if return_models is True) – dictionary of models
errors (list, always returned) – list of model names that could not be created

export_model_series_to_csv(names: list | str | None = None, exportdir: str = '.', exportmeta: bool = True)[source]

Export model time series to csv files.

Parameters:

names (Optional[Union[list, str]], optional) – names of models to export, by default None, which uses retrieves all models from database
exportdir (str, optional) – directory to export csv files to, default is current directory
exportmeta (bool, optional) – export metadata for all time series as csv file, default is True

classmethod from_zip(fname: str, conn: BaseConnector | None = None, storename: str | None = None, progressbar: bool = True)[source]

Load PastaStore from zipfile.

Parameters:

fname (str) – pathname of zipfile
conn (Connector object, optional) – connector for storing loaded data, default is None which creates a DictConnector. This Connector does not store data on disk.
storename (str, optional) – name of the PastaStore, by default None, which defaults to the name of the Connector.
progressbar (bool, optional) – show progressbar, by default True

Returns:

return PastaStore containing data from zipfile

Return type:

pastastore.PastaStore

Method to obtain the distances in meters between the oseries and stresses.

Parameters:

oseries (str or list of str) – name(s) of the oseries
stresses (str or list of str) – name(s) of the stresses
kind (str, list of str) – string or list of strings representing which kind(s) of stresses to consider

Returns:

distances – Pandas DataFrame with the distances between the oseries (index) and the stresses (columns).

Return type:

pandas.DataFrame

get_model_timeseries_names(modelnames: list | str | None = None, dropna: bool = True, progressbar: bool = True) → DataFrame | Series[source]

Get time series names contained in model.

Parameters:

modelnames (Optional[Union[list, str]], optional) – list or name of models to get time series names for, by default None which will use all modelnames
dropna (bool, optional) – drop stresses from table if stress is not included in any model, by default True
progressbar (bool, optional) – show progressbar, by default True

Returns:

structure – returns DataFrame with oseries name per model, and a flag indicating whether a stress is contained within a time series model.

Return type:

pandas.DataFrame

get_nearest_oseries(names: list | str | None = None, n: int = 1, maxdist: float | None = None) → DataFrame | Series[source]

Method to obtain the nearest (n) oseries.

Parameters:

names (str or list of str) – string or list of strings with the name(s) of the oseries
n (int) – number of oseries to obtain
maxdist (float, optional) – maximum distance to consider

Returns:

list with the names of the oseries.

Return type:

oseries

Method to obtain the nearest (n) stresses of a specific kind.

Parameters:

oseries (str) – string with the name of the oseries
stresses (str or list of str) – string with the name of the stresses
kind (str, list of str, optional) – string or list of str with the name of the kind(s) of stresses to consider
n (int) – number of stresses to obtain
maxdist (float, optional) – maximum distance to consider

Returns:

list with the names of the stresses.

Return type:

stresses

get_oseries_distances(names: list | str | None = None) → DataFrame | Series[source]

Method to obtain the distances in meters between the oseries.

Parameters:: names (str or list of str) – names of the oseries to calculate distances between
Returns:: distances – Pandas DataFrame with the distances between the oseries
Return type:: pandas.DataFrame

Get model parameters. NaN-values are returned when the parameters are not present in the model or the model is not optimized.

Parameters:

parameters (list of str, optional) – names of the parameters, by default None which uses all parameters from each model
modelnames (str or list of str, optional) – name(s) of model(s), by default None in which case all models are used
param_value (str, optional) – which column to use from the model parameters dataframe, by default “optimal” which retrieves the optimized parameters.
progressbar (bool, optional) – show progressbar, default is False
ignore_errors (bool, optional) – ignore errors when True, i.e. when non-existent model is encountered in modelnames, by default False

Returns:

p – DataFrame containing the parameters (columns) per model (rows)

Return type:

pandas.DataFrame

get_signatures(signatures=None, names=None, libname='oseries', progressbar=False, ignore_errors=False)[source]

Get groundwater signatures. NaN-values are returned when the signature could not be computed.

Parameters:

signatures (list of str, optional) – list of groundwater signatures to compute, if None all groundwater signatures in ps.stats.signatures.__all__ are used, by default None
names (str, list of str, or None, optional) – names of the time series, by default None which uses all the time series in the library
libname (str) – name of the library containing the time series (‘oseries’ or ‘stresses’), by default “oseries”
progressbar (bool, optional) – show progressbar, by default False
ignore_errors (bool, optional) – ignore errors when True, i.e. when non-existent timeseries is encountered in names, by default False

Returns:

signatures_df – DataFrame containing the signatures (columns) per time series (rows)

Return type:

pandas.DataFrame

Get model statistics.

Parameters:

statistics (str or list of str) – statistic or list of statistics to calculate, e.g. [“evp”, “rsq”, “rmse”], for a full list see pastas.modelstats.Statistics.ops.
modelnames (list of str, optional) – modelnames to calculates statistics for, by default None, which uses all models in the store
progressbar (bool, optional) – show progressbar, by default False
ignore_errors (bool, optional) – ignore errors when True, i.e. when trying to calculate statistics for non-existent model in modelnames, default is False
**kwargs – any arguments that can be passed to the methods for calculating statistics

Returns:

s

Return type:

pandas.DataFrame

get_tmin_tmax(libname, names=None, progressbar=False)[source]

Get tmin and tmax for time series.

Parameters:

libname (str) – name of the library containing the time series (‘oseries’ or ‘stresses’)
names (str, list of str, or None, optional) – names of the time series, by default None which uses all the time series in the library
progressbar (bool, optional) – show progressbar, by default False

Returns:

tmintmax – Dataframe containing tmin and tmax per time series

Return type:

pd.dataframe

model_results(mls: Model | list | str | None = None, progressbar: bool = True)[source]

Get pastas model results.

Parameters:

mls (list of str, optional) – list of model names, by default None which means results for all models will be calculated
progressbar (bool, optional) – show progressbar, by default True

Returns:

results – dataframe containing parameters and other statistics for each model

Return type:

pd.DataFrame

Raises:

ModuleNotFoundError – if the art_tools module is not available

search(libname: str, s: list | str | None = None, case_sensitive: bool = True, sort=True)[source]

Search for names of time series or models starting with s.

Parameters:

libname (str) – name of the library to search in
s (str, lst) – find names with part of this string or strings in list
case_sensitive (bool, optional) – whether search should be case sensitive, by default True
sort (bool, optional) – sort list of names

Returns:

matches – list of names that match search result

Return type:

list

solve_models(mls: Model | list | str | None = None, report: bool = False, ignore_solve_errors: bool = False, store_result: bool = True, progressbar: bool = True, **kwargs) → None[source]

Solves the models in the store.

Parameters:

mls (list of str, optional) – list of model names, if None all models in the pastastore are solved.
report (boolean, optional) – determines if a report is printed when the model is solved, default is False
ignore_solve_errors (boolean, optional) – if True, errors emerging from the solve method are ignored, default is False which will raise an exception when a model cannot be optimized
store_result (bool, optional) – if True save optimized models, default is True
progressbar (bool, optional) – show progressbar, default is True
**kwargs – arguments are passed to the solve method.

to_zip(fname: str, overwrite=False, progressbar: bool = True)[source]

Write data to zipfile.

Parameters:

fname (str) – name of zipfile
overwrite (bool, optional) – if True, overwrite existing file
progressbar (bool, optional) – show progressbar, by default True

Plots

class pastastore.plotting.Plots(pstore)[source]

Plot class for Pastastore.

Allows plotting of time series and data availability.

static _data_availability(series, names=None, intervals=None, ignore=('second', 'minute', '14 days'), ax=None, cax=None, normtype='log', cmap='viridis_r', set_yticks=False, figsize=(10, 8), dropna=True, **kwargs)[source]

Plot the data-availability for a list of time series.

Parameters:

libname (list of pandas.Series) – list of series to plot data availability for
names (list, optional) – specify names of series, default is None in which case names will be taken from series themselves.
kind (str, optional) – if library is stresses, kind can be specified to obtain only stresses of a specific kind
intervals (dict, optional) – A dict with frequencies as keys and number of seconds as values
ignore (list, optional) – A list with frequencies in intervals to ignore
ax (matplotlib Axes, optional) – pass axes object to plot data availability on existing figure. by default None, in which case a new figure is created
cax (matplotlib Axes, optional) – pass object axes to plot the colorbar on. by default None, which gives default Maptlotlib behavior
normtype (str, optional) – Determines the type of color normalisations, default is ‘log’
cmap (str, optional) – A reference to a matplotlib colormap
set_yticks (bool, optional) – Set the names of the series as yticks
figsize (tuple, optional) – The size of the new figure in inches (h,v)
progressbar (bool) – Show progressbar
dropna (bool) – Do not show NaNs as available data
kwargs (dict, optional) – Extra arguments are passed to matplotlib.pyplot.subplots()

Returns:

ax – The axes in which the data-availability is plotted

Return type:

matplotlib Axes

_timeseries(libname, names=None, ax=None, split=False, figsize=(10, 5), progressbar=True, show_legend=True, labelfunc=None, legend_kwargs=None, **kwargs)[source]

Internal method to plot time series from pastastore.

Parameters:

libname (str) – name of the library to obtain time series from (oseries or stresses)
names (list of str, optional) – list of time series names to plot, by default None
ax (matplotlib.Axes, optional) – pass axes object to plot on existing axes, by default None, which creates a new figure
split (bool, optional) – create a separate subplot for each time series, by default False. A maximum of 20 time series is supported when split=True.
figsize (tuple, optional) – figure size, by default (10, 5)
progressbar (bool, optional) – show progressbar when loading time series from store, by default True
show_legend (bool, optional) – show legend, default is True.
labelfunc (callable, optional) – function to create custom labels, function should take name of time series as input
legend_kwargs (dict, optional) – additional arguments to pass to legend

Returns:

ax – axes handle

Return type:

matplotlib.Axes

Raises:

ValueError – split=True is only supported if there are less than 20 time series to plot.

compare_models(modelnames, ax=None, **kwargs)[source]

cumulative_hist(statistic='rsq', modelnames=None, extend=False, ax=None, figsize=(6, 6), label=None, legend=True)[source]

Plot a cumulative step histogram for a model statistic.

Parameters:

statistic (str) – name of the statistic, e.g. “evp” or “rmse”, by default “rsq”
modelnames (list of str, optional) – modelnames to plot statistic for, by default None, which uses all models in the store
extend (bool, optional) – force extend the stats Series with a dummy value to move the horizontal line outside figure bounds. If True the results are skewed a bit, especially if number of models is low.
ax (matplotlib.Axes, optional) – axes to plot histogram, by default None which creates an Axes
figsize (tuple, optional) – figure size, by default (6,6)
label (str, optional) – label for the legend, by default None, which shows the number of models
legend (bool, optional) – show legend, by default True

Returns:

ax – The axes in which the cumulative histogram is plotted

Return type:

matplotlib Axes

data_availability(libname, names=None, kind=None, intervals=None, ignore=('second', 'minute', '14 days'), ax=None, cax=None, normtype='log', cmap='viridis_r', set_yticks=False, figsize=(10, 8), progressbar=True, dropna=True, **kwargs)[source]

Plot the data-availability for multiple time series in pastastore.

Parameters:

libname (str) – name of library to get time series from (oseries or stresses)
names (list, optional) – specify names in a list to plot data availability for certain time series
kind (str, optional) – if library is stresses, kind can be specified to obtain only stresses of a specific kind
intervals (dict, optional) – A dict with frequencies as keys and number of seconds as values
ignore (list, optional) – A list with frequencies in intervals to ignore
ax (matplotlib Axes, optional) – pass axes object to plot data availability on existing figure. by default None, in which case a new figure is created
cax (matplotlib Axes, optional) – pass object axes to plot the colorbar on. by default None, which gives default Maptlotlib behavior
normtype (str, optional) – Determines the type of color normalisations, default is ‘log’
cmap (str, optional) – A reference to a matplotlib colormap
set_yticks (bool, optional) – Set the names of the series as yticks
figsize (tuple, optional) – The size of the new figure in inches (h,v)
progressbar (bool) – Show progressbar
dropna (bool) – Do not show NaNs as available data
kwargs (dict, optional) – Extra arguments are passed to matplotlib.pyplot.subplots()

Returns:

ax – The axes in which the data-availability is plotted

Return type:

matplotlib Axes

oseries(names=None, ax=None, split=False, figsize=(10, 5), show_legend=True, labelfunc=None, legend_kwargs=None, **kwargs)[source]

Plot oseries.

Parameters:

names (list of str, optional) – list of oseries names to plot, by default None, which loads all oseries from store
ax (matplotlib.Axes, optional) – pass axes object to plot oseries on existing figure, by default None, in which case a new figure is created
split (bool, optional) – create a separate subplot for each time series, by default False. A maximum of 20 time series is supported when split=True.
figsize (tuple, optional) – figure size, by default (10, 5)
show_legend (bool, optional) – show legend, default is True.
labelfunc (callable, optional) – function to create custom labels, function should take name of time series as input
legend_kwargs (dict, optional) – additional arguments to pass to legend

Returns:

ax – axes handle

Return type:

matplotlib.Axes

stresses(names=None, kind=None, ax=None, split=False, figsize=(10, 5), show_legend=True, labelfunc=None, legend_kwargs=None, **kwargs)[source]

Plot stresses.

Parameters:

names (list of str, optional) – list of oseries names to plot, by default None, which loads all oseries from store
kind (str, optional) – only plot stresses of a certain kind, by default None, which includes all stresses
ax (matplotlib.Axes, optional) – pass axes object to plot oseries on existing figure, by default None, in which case a new figure is created
split (bool, optional) – create a separate subplot for each time series, by default False. A maximum of 20 time series is supported when split=True.
figsize (tuple, optional) – figure size, by default (10, 5)
show_legend (bool, optional) – show legend, default is True.
labelfunc (callable, optional) – function to create custom labels, function should take name of time series as input
legend_kwargs (dict, optional) – additional arguments to pass to legend

Returns:

ax – axes handle

Return type:

matplotlib.Axes

Maps

class pastastore.plotting.Maps(pstore)[source]

Map Class for PastaStore.

Allows plotting locations and model statistics on maps.

Usage

Example usage of the maps methods: :

>> > ax = pstore.maps.oseries() # plot oseries locations >> > pstore.maps.add_background_map(ax) # add background map

static _list_contextily_providers()[source]

List contextily providers.

Taken from contextily notebooks.

Returns:: providers – dictionary containing all providers. See keys for names that can be passed as map_provider arguments.
Return type:: dict

static _plotmap_dataframe(df, x='x', y='y', column=None, colorbar=True, ax=None, figsize=(10, 8), **kwargs)[source]

Internal method for plotting dataframe with point locations.

Can be called directly for more control over plot characteristics.

Parameters:

df (pandas.DataFrame) – DataFrame containing coordinates and data to plot, with index providing names for each location
x (str, optional) – name of the column with x - coordinate data, by default “x”
y (str, optional) – name of the column with y - coordinate data, by default “y”
column (str, optional) – name of the column containing data used for determining the color of each point, by default None (all one color)
colorbar (bool, optional) – show colorbar, only if column is provided, by default True
ax (matplotlib Axes) – axes handle to plot dataframe, optional, default is None which creates a new figure
figsize (tuple, optional) – figure size, by default(10, 8)
**kwargs – dictionary containing keyword arguments for ax.scatter, by default None

Returns:

ax (matplotlib.Axes) – axes object, returned if ax is None
sc (scatter handle) – scatter plot handle, returned if ax is not None

static add_background_map(ax, proj='epsg:28992', map_provider='OpenStreetMap.Mapnik', **kwargs)[source]

Add background map to axes using contextily.

Parameters:

ax (matplotlib.Axes) – axes to add background map to
map_provider (str, optional) – name of map provider, see contextily.providers for options. Default is ‘OpenStreetMap.Mapnik’
proj (pyproj.Proj or str, optional) – projection for background map, default is ‘epsg:28992’ (RD Amersfoort, a projection for the Netherlands)

static add_labels(df, ax, adjust=False, objects=None, **kwargs)[source]

Add labels to points on plot.

Uses dataframe index to label points.

Parameters:

df (pd.DataFrame) – DataFrame containing x, y - data. Index is used as label
ax (matplotlib.Axes) – axes object to label points on
adjust (bool) – automated smart label placement using adjustText
objects (list of matplotlib objects) – use to avoid labels overlapping markers
**kwargs – keyword arguments to ax.annotate or adjusttext

model(ml, label=True, metadata_source='model', offset=0.0, ax=None, figsize=(10, 10), backgroundmap=False)[source]

Plot oseries and stresses from one model on a map.

Parameters:

ml (str or pastas.Model) – pastas model or name of pastas model to plot on map
label (bool, optional, default is True) – add labels to points on map
metadata_source (str, optional) – whether to obtain metadata from model Timeseries or from metadata in pastastore(“store”), default is “model”
offset (float, optional) – add offset to current extent of model time series, useful for zooming out around models
ax (matplotlib.Axes, optional) – axes handle, if not provided a new figure is created.
figsize (tuple, optional) – figsize, default is (10, 10)
backgroundmap (bool, optional) – if True, add background map (default CRS is EPSG:28992) with default tiles by OpenStreetMap.Mapnik. Default option is False.

Returns:

ax – axis handle of the resulting figure

Return type:

axes object

See also

self.add_background_map

models(labels=True, adjust=False, figsize=(10, 8), backgroundmap=False, **kwargs)[source]

Plot model locations on map.

Parameters:

labels (bool, optional) – label models, by default True
adjust (bool, optional) – automated smart label placement using adjustText, by default False
ax (matplotlib.Axes, optional) – axes handle, if not provided a new figure is created.
figsize (tuple, optional) – figure size, by default(10, 8)
backgroundmap (bool, optional) – if True, add background map (default CRS is EPSG:28992) with default tiles by OpenStreetMap.Mapnik. Default option is False.

Returns:

ax – axes object

Return type:

matplotlib.Axes

See also

self.add_background_map

modelstat(statistic, modelnames=None, label=True, adjust=False, cmap='viridis', norm=None, vmin=None, vmax=None, figsize=(10, 8), backgroundmap=False, **kwargs)[source]

Plot model statistic on map.

Parameters:

statistic (str) – name of the statistic, e.g. “evp” or “aic”
modelnames (list of str, optional) – list of modelnames to include
label (bool, optional) – label points, by default True
adjust (bool, optional) – automated smart label placement using adjustText, by default False
cmap (str or colormap, optional) – (name of) the colormap, by default “viridis”
norm (norm, optional) – normalization for colorbar, by default None
vmin (float, optional) – vmin for colorbar, by default None
vmax (float, optional) – vmax for colorbar, by default None
ax (matplotlib.Axes, optional) – axes handle, if not provided a new figure is created.
figsize (tuple, optional) – figuresize, by default(10, 8)
backgroundmap (bool, optional) – if True, add background map (default CRS is EPSG:28992) with default tiles by OpenStreetMap.Mapnik. Default option is False.

Returns:

ax – axes object

Return type:

matplotlib.Axes

See also

self.add_background_map

oseries(names=None, extent=None, labels=True, adjust=False, figsize=(10, 8), backgroundmap=False, label_kwargs=None, **kwargs)[source]

Plot oseries locations on map.

Parameters:

names (list, optional) – oseries names, by default None which plots all oseries locations
extent (list of float, optional) – plot only oseries within extent [xmin, xmax, ymin, ymax]
labels (bool or str, optional) – label models, by default True, if passed as “grouped”, only the first label for each x,y-location is shown.
adjust (bool, optional) – automated smart label placement using adjustText, by default False
figsize (tuple, optional) – figure size, by default(10, 8)
backgroundmap (bool, optional) – if True, add background map (default CRS is EPSG:28992) with default tiles by OpenStreetMap.Mapnik. Default option is False.
label_kwargs (dict, optional) – dictionary with keyword arguments to pass to add_labels method

Returns:

ax – axes object

Return type:

matplotlib.Axes

See also

self.add_background_map

stresses(names=None, kind=None, extent=None, labels=True, adjust=False, figsize=(10, 8), backgroundmap=False, label_kwargs=None, **kwargs)[source]

Plot stresses locations on map.

Parameters:

names (list of str, optional) – list of names to plot
kind (str, optional) – if passed, only plot stresses of a specific kind, default is None which plots all stresses.
extent (list of float, optional) – plot only stresses within extent [xmin, xmax, ymin, ymax]
labels (bool, optional) – label models, by default True
adjust (bool, optional) – automated smart label placement using adjustText, by default False
ax (matplotlib.Axes, optional) – axes handle, if not provided a new figure is created.
figsize (tuple, optional) – figure size, by default(10, 8)
backgroundmap (bool, optional) – if True, add background map (default CRS is EPSG:28992) with default tiles by OpenStreetMap.Mapnik. Default option is False.
label_kwargs (dict, optional) – dictionary with keyword arguments to pass to add_labels method

Returns:

ax – axes object

Return type:

matplotlib.Axes

See also

self.add_background_map

stresslinks(kinds=None, model_names=None, color_lines=False, alpha=0.4, ax=None, figsize=(10, 8), legend=True, labels=False, adjust=False, backgroundmap=False)[source]

Create a map linking models with their stresses.

Parameters:

kinds (list, optional) – kinds of stresses to plot, defaults to None, which selects all kinds.
model_names (list, optional) – list of model names to plot, substrings of model names are also accepted, defaults to None, which selects all models.
color_lines (bool, optional) – if True, connecting lines have the same colors as the stresses, defaults to False, which uses a black line.
alpha (float, optional) – alpha value for the connecting lines, defaults to 0.4.
ax (matplotlib.Axes, optional) – axes handle, if not provided a new figure is created.
figsize (tuple, optional) – figure size, by default (10, 8)
legend (bool, optional) – create a legend for all unique kinds, defaults to True.
labels (bool, optional) – add labels for stresses and oseries, defaults to False.
adjust (bool, optional) – automated smart label placement using adjustText, by default False
backgroundmap (bool, optional) – if True, add background map (default CRS is EPSG:28992) with default tiles by OpenStreetMap.Mapnik. Default option is False.

Returns:

ax – axis handle of the resulting figure

Return type:

axes object

See also

self.add_background_map

Yaml

class pastastore.yaml_interface.PastastoreYAML(pstore)[source]

Class for reading/writing Pastas models in YAML format.

This class provides a more human-readable form of Pastas models in comparison to Pastas default .pas (JSON) files. The goal is to provide users with a simple mini-language to quickly build/test different model structures. A PastaStore is required as input, which contains existing models or time series required to build new models. This class also introduces some shortcuts to simplify building models. Shortcuts include the option to pass ‘nearest’ as the name of a stress, which will automatically select the closest stress of a particular type. Other shortcuts include certain default options when certain information is not listed in the YAML file, that will work well in many cases.

Usage

Instantiate the PastastoreYAML class:

pyaml = PastastoreYAML(pstore)

Export a Pastas model to a YAML file:

pyaml.export_model_to_yaml(ml)

Load a Pastas model from a YAML file:

models = pyaml.load_yaml("my_first_model.yaml")

Example YAML file using ‘nearest’:

my_first_model:  # this is the name of the model
  oseries: "oseries1"  # name of oseries stored in PastaStore
  stressmodels:
      recharge:  # recognized as RechargeModel by name
        prec: "nearest"  # use nearest stress with kind="prec"
        evap: "EV24_DEELEN"  # specific station
      river:
        stress: "nearest riv"  # nearest stress with kind="riv"
      wells:
        stress: "nearest 3"  # nearest 3 stresses with kind="well"
        stressmodel: WellModel  # provide StressModel type

static export_model(ml: Model | dict, outdir: str | None = '.', minimal_yaml: bool | None = False, use_nearest: bool | None = False)[source]

Write single pastas model to YAML file.

Parameters:

ml (ps.Model or dict) – pastas model instance or dictionary representing a pastas model
outdir (str, optional) – path to output directory, by default “.” (current directory)
minimal_yaml (bool, optional) – reduce yaml file to include the minimum amount of information that will still construct a model. Users are warned, using this option does not guarantee the same model will be constructed as the one that was exported! Default is False.
use_nearest (bool, optional) – if True, replaces time series with “nearest <kind>”, filling in kind where possible. Warning! This does not check whether the time series are actually the nearest ones! Only used when minimal_yaml=True. Default is False.

Export (stored) models to yaml file(s).

Parameters:

models (list of ps.Model or dict, optional) – pastas Models to write to yaml file(s), if not provided, uses modelnames to collect stored models to export.
modelnames (list of str, optional) – list of model names to export, by default None, which uses all stored models.
outdir (str, optional) – path to output directory, by default “.” (current directory)
minimal_yaml (bool, optional) – reduce yaml file to include the minimum amount of information that will still construct a model. Users are warned, using this option does not guarantee the same model will be constructed as the one that was exported! Default is False.
use_nearest (bool, optional) – if True, replaces time series with “nearest <kind>”, filling in kind where possible. Warning! This does not check whether the time series are actually the nearest ones! Only used when minimal_yaml=True. Default is False.
split (bool, optional) – if True, split into separate yaml files, otherwise store all in the same file. The model names are used as file names.
filename (str, optional) – filename for YAML file, only used if split=False

Export store models grouped per oseries (location) to YAML file(s).

Note: The oseries names are used as file names.

Parameters:

oseries (list of str, optional) – list of oseries (location) names, by default None, which uses all stored oseries for which there are models.
outdir (str, optional) – path to output directory, by default “.” (current directory)
minimal_yaml (bool, optional) – reduce yaml file to include the minimum amount of information that will still construct a model. Users are warned, using this option does not guarantee the same model will be constructed as the one that was exported! Default is False.
use_nearest (bool, optional) – if True, replaces time series with “nearest <kind>”, filling in kind where possible. Warning! This does not check whether the time series are actually the nearest ones! Only used when minimal_yaml=True. Default is False.

load(fyaml: str) → List[Model][source]

Load Pastas YAML file.

Note: currently supports RechargeModel, StressModel and WellModel.

Parameters:

fyaml (str) – path to file

Returns:

models – list containing pastas model(s)

Return type:

list

Raises:

ValueError – if insufficient information is provided to construct pastas model
NotImplementedError – if unsupported stressmodel is encountered

pastastore.yaml_interface.reduce_to_minimal_dict(d, keys=None)[source]

Reduce pastas model dictionary to a minimal form.

This minimal form strives to keep the minimal information that still allows a model to be constructed. Users are warned, reducing a model dictionary with this function can lead to a different model than the original!

Parameters:

d (dict) – pastas model in dictionary form
keys (list, optional) – list of keys to keep, by default None, which defaults to: [“name”, “oseries”, “settings”, “tmin”, “tmax”, “noise”, “stressmodels”, “rfunc”, “stress”, “prec”, “evap”, “stressmodel”]

pastastore.yaml_interface.replace_ts_with_name(d, nearest=False)[source]

Replace time series dict with its name in pastas model dict.

Parameters:

d (dict) – pastas model dictionary
nearest (bool, optional) – replace time series with “nearest” option. Warning, this does not check whether the time series are actually the nearest ones!

Util

exception pastastore.util.ItemInLibraryException[source]

pastastore.util.compare_models(ml1, ml2, stats=None, detailed_comparison=False)[source]

Compare two Pastas models.

Parameters:

ml1 (pastas.Model) – first model to compare
ml2 (pastas.Model) – second model to compare
stats (list of str, optional) – if provided compare these model statistics
detailed_comparison (bool, optional) – if True return DataFrame containing comparison details, by default False which returns True if models are equivalent or False if they are not

Returns:

returns True if models are equivalent when detailed_comparison=True else returns DataFrame containing comparison details.

Return type:

bool or pd.DataFrame

pastastore.util.copy_database(conn1, conn2, libraries: List[str] | None = None, overwrite: bool = False, progressbar: bool = False) → None[source]

Copy libraries from one database to another.

Parameters:

conn1 (pastastore.*Connector) – source Connector containing link to current database containing data
conn2 (pastastore.*Connector) – destination Connector with link to database to which you want to copy
libraries (Optional[List[str]], optional) – list of str containing names of libraries to copy, by default None, which copies all libraries: [‘oseries’, ‘stresses’, ‘models’]
overwrite (bool, optional) – overwrite data in destination database, by default False
progressbar (bool, optional) – show progressbars, by default False

Raises:

ValueError – if library name is not understood

pastastore.util.delete_arctic_connector(conn=None, connstr: str | None = None, name: str | None = None, libraries: List[str] | None = None) → None[source]

Delete libraries from arctic database.

Parameters:

conn (pastastore.ArcticConnector) – ArcticConnector object
connstr (str, optional) – connection string to the database
name (str, optional) – name of the database
libraries (Optional[List[str]], optional) – list of library names to delete, by default None which deletes all libraries

pastastore.util.delete_arcticdb_connector(conn=None, uri: str | None = None, name: str | None = None, libraries: List[str] | None = None) → None[source]

Delete libraries from arcticDB database.

Parameters:

conn (pastastore.ArcticDBConnector) – ArcticDBConnector object
uri (str, optional) – uri connection string to the database
name (str, optional) – name of the database
libraries (Optional[List[str]], optional) – list of library names to delete, by default None which deletes all libraries

pastastore.util.delete_pastastore(pstore, libraries: List[str] | None = None) → None[source]

Delete libraries from PastaStore.

Note

This deletes the original PastaStore object. To access data that has not been deleted, it is recommended to create a new PastaStore object with the same Connector settings. This also creates new empty libraries if they were deleted.

Parameters:

pstore (pastastore.PastaStore) – PastaStore object to delete (from)
libraries (Optional[List[str]], optional) – list of library names to delete, by default None which deletes all libraries

Raises:

TypeError – when Connector type is not recognized

pastastore.util.delete_pystore_connector(conn=None, path: str | None = None, name: str | None = None, libraries: List[str] | None = None) → None[source]

Delete libraries from pystore.

Parameters:

conn (PystoreConnector, optional) – PystoreConnector object
path (str, optional) – path to pystore
name (str, optional) – name of the pystore
libraries (Optional[List[str]], optional) – list of library names to delete, by default None which deletes all libraries

pastastore.util.frontiers_aic_select(pstore, modelnames: List[str] | None = None, oseries: List[str] | None = None, full_output: bool = False) → DataFrame[source]

Select the best model structure based on the minimum AIC.

As proposed by Brakenhoff et al. 2022 [bra_2022].

Parameters:

pstore (pastastore.PastaStore) – reference to a PastaStore
modelnames (list of str) – list of model names (that pass reliability criteria)
oseries (list of oseries) – list of locations for which to select models, note that this uses all models associated with a specific location.
full_output (bool, optional) – if set to True, returns a DataFrame including all models per location and their AIC values

Returns:

DataFrame with selected best model per location based on the AIC, or a DataFrame containing statistics for each of the models per location

Return type:

pandas.DataFrame

References

[bra_2022]

Brakenhoff, D.A., Vonk M.A., Collenteur, R.A., van Baar, M.,

Bakker, M.: Application of Time Series Analysis to Estimate Drawdown From Multiple Well Fields. Front. Earth Sci., 14 June 2022 doi:10.3389/feart.2022.907609

pastastore.util.frontiers_checks(pstore, modelnames: List[str] | None = None, oseries: List[str] | None = None, check1_rsq: bool = True, check1_threshold: float = 0.7, check2_autocor: bool = True, check2_test: str = 'runs', check2_pvalue: float = 0.05, check3_tmem: bool = True, check3_cutoff: float = 0.95, check4_gain: bool = True, check5_parambounds: bool = False, csv_dir: str | None = None) → DataFrame[source]

Check models in a PastaStore to see if they pass reliability criteria.

The reliability criteria are taken from Brakenhoff et al. 2022 [bra_2022]. These criteria were applied in a region with recharge, river levels and pumping wells as stresses. This is by no means an exhaustive list of reliability criteria but might serve as a reasonable starting point for model diagnostic checking.

Parameters:

pstore (pastastore.PastaStore) – reference to a PastaStore
modelnames (list of str, optional) – list of model names to consider, if None checks ‘oseries’, if both are None, all stored models will be checked
oseries (list of str, optional) – list of oseries to consider, corresponding models will be picked up from pastastore. If None, uses all stored models are checked.
check1 (bool, optional) – check if model fit is above a threshold of the coefficient of determination $R^2$ , by default True
check1_threshold (float, optional) – threshold of the $R^2$ fit statistic, by default 0.7
check2 (bool, optional) – check if the noise of the model has autocorrelation with statistical test, by default True
check2_test (str, optional) – statistical test for autocorrelation. Available options are Runs test “runs”, Stoffer-Toloi “stoffer” or “both”, by default “runs”
check2_pvalue (float, optional) – p-value for the statistical test to define the confindence interval, by default 0.05
check3 (bool, optional) – check if the length of the response time is within the calibration period, by default True
check3_cutoff (float, optional) – the cutoff of the response time, by default 0.95
check4 (bool, optional) – check if the uncertainty of the gain, by default True
check5 (bool, optional) – check if parameters hit parameter bounds, by default False
csv_dir (string, optional) – directory to store CSV file with overview of checks for every model, by default None which will not store results

Returns:

df – dataFrame with all models and whether or not they pass the reliability checks

Return type:

pandas.DataFrame

References

[bra_2022]

Brakenhoff, D.A., Vonk M.A., Collenteur, R.A., van Baar, M., Bakker, M.: Application of Time Series Analysis to Estimate Drawdown From Multiple Well Fields. Front. Earth Sci., 14 June 2022 doi:10.3389/feart.2022.907609

Remove invalid characters from string or dictionary keys.

Parameters:

s (str, optional) – remove invalid characters from string
d (dict, optional) – remove invalid characters from keys from dictionary
replace_space (str, optional) – replace spaces by this character, by default “_”
deletechars (str, optional) – a string combining invalid characters, by default None

Returns:

string or dict with invalid characters removed

Return type:

str, dict