pypfilt.crps

This module provides functions for calculating Continuous Ranked Probability Scores. CRPS is a measure of how well a forecast distribution predicts the true values.

You can evaluate forecast performance against future observations by:

  1. Simulating observations for each particle, using the SimulatedObs table;

  2. Loading the simulated observations with load_dataset();

  3. Loading the future observations (once available) with the observation model’s from_file() method; and

  4. Calculating the CRPS scores with simulated_obs_crps().

pypfilt.crps.simulated_obs_crps(true_obs, sim_obs)

Calculate CRPS scores for simulated observations, such as those recorded by the SimulatedObs table, against observed values.

The returned array has fields: 'time', 'fs_time', and 'score'.

Parameters:

  • true_obs – The table of recorded observations; this must contain the fields 'time' and 'value’.

  • sim_obs – The table of simulated observations; this must contain the fields 'fs_time', 'time', and 'value'.

Raises:

ValueError – if true_obs or sim_obs do not contain all of the required fields.

pypfilt.crps.crps_edf_scalar(true_value, sample_values)

Calculate the CRPS for samples drawn from a predictive distribution for a single value, using the probability weighted moment CRPS estimator.

Parameters:

  • true_value – The (scalar) value that was observed.

  • sample_values – Samples from the predictive distribution (a 1-D array).

See equation (ePWM) in Zamo and Naveau, 2020.

Note that we use a different definition of the ensemble quantiles, \(\frac{i - 0.5}{m} : i \in {1, \dots, m}\), as presented in Bröcker 2012 and noted in Ferro 2013.

pypfilt.crps.crps_sample(true_values, samples_table)

Calculate the CRPS score for a table of samples drom from predictive distributions for multiple values, using the empirical distribution function defined by the provided samples.

Parameters:

  • true_values – A 1-D array of observed values.

  • samples_table – A 2-D array of samples, where each row contains the samples for the corresponding value in true_values.