6.9. pypfilt.crps

This module provides functions for calculating Continuous Ranked Probability Scores. CRPS is a measure of how well a forecast distribution predicts the true values.

You can evaluate forecast performance against future observations by:

  1. Simulating observations for each particle, using the SimulatedObs table;
  2. Loading the simulated observations with load_dataset();
  3. Loading the future observations (once available) with the observation model’s from_file() method; and
  4. Calculating the CRPS scores with simulated_obs_crps().
pypfilt.crps.simulated_obs_crps(true_obs, sim_obs)

Calculate CRPS scores for simulated observations, such as those recorded by the SimulatedObs table, against observed values.

The returned array has fields: 'date', 'fs_date', and 'score'.

Parameters:
  • true_obs – The table of recorded observations; this must contain the fields 'date' and 'value’.
  • sim_obs – The table of simulated observations; this must contain the fields 'fs_date', 'date', and 'value'.
Raises:

ValueError – if true_obs or sim_obs do not contain all of the required fields.
pypfilt.crps.crps_edf_scalar(true_value, samples)

Calculate the CRPS score for samples drawn from a predictive distribution for a single value, using the empirical distribution function defined by the provided samples.

Parameters:
  • true_value – The (scalar) value that was observed.
  • samples – Samples from the predictive distribution (a 1-D array).
pypfilt.crps.crps_sample(true_values, samples_table)

Calculate the CRPS score for a table of samples drom from predictive distributions for multiple values, using the empirical distribution function defined by the provided samples.

Parameters:
  • true_values – A 1-D array of observed values.
  • samples_table – A 2-D array of samples, where each row contains the samples for the corresponding value in true_values.