5.10. pypfilt.io¶
The pypfilt.io
module provides functions for reading tabular data from
data files.
5.10.1. Reading tabular data¶
-
pypfilt.io.
read_table
(path, columns, comment='#', encoding='utf-8')¶ Read data from a space-delimited text file with column headers defined in the first non-comment line.
Parameters: - path – The path to the data file.
- columns – The columns to read from the data file, represented as a
sequence of
(name, type)
tuples wheretype
must be a NumPy scalar type, or(name, type, converter)
tuples whereconverter
is a function that converts the column string into the desired value. - comment – The characters, or list of characters, that indicate the start of a single-line comment.
- encoding – The name of the encoding used to decode the file content.
Examples: >>> from pypfilt.io import date_column, read_table >>> import numpy as np >>> import datetime >>> path = "input_data.ssv" >>> with open(path, 'w') as f: ... _ = f.write('date value\n') ... _ = f.write('2020-01-01 1\n') ... _ = f.write('2020-01-02 3\n') ... _ = f.write('2020-01-03 5\n') >>> columns = [date_column('date'), ('value', np.int_)] >>> data = read_table(path, columns) >>> isinstance(data['date'][0], datetime.datetime) True >>> observations = [{'date': row['date'], 'value': row['value']} ... for row in data]
-
pypfilt.io.
date_column
(name, fmt='%Y-%m-%d')¶ Return a
(name, type, converter)
tuple that can be used withread_table()
to convert a column intodatetime.datetime
values.Note
Where dates are used for observation times, they should be represented as
datetime.datetime
values, not asdatetime.date
values. This is why this function returns a converter that returnsdatetime.datetime
values.Parameters: - name (str) – The column name in the data file.
- fmt (str) – The date format used to parse the column values.
-
pypfilt.io.
datetime_column
(name, fmt='%Y-%m-%dT%H:%M:%S')¶ Return a
(name, type, converter)
tuple that can be used withread_table()
to convert a column intodatetime.datetime
values.Parameters: - name (str) – The column name in the data file.
- fmt (str) – The datetime format used to parse the column values.
5.10.2. Lookup tables¶
The pypfilt.io
module also provides lookup tables, which are used to
retrieve time-indexed values (e.g., time-varying model inputs).
-
pypfilt.io.
read_lookup_table
(path, time, dtype='f8', comment='#', encoding='utf-8')¶ Read time-indexed data from a space-delimited text file with column headers defined in the first non-comment line.
Parameters: - path – The path to the data file.
- time (pypfilt.time.Time) – The time scale.
- dtype – The type of the lookup values.
- comment – The characters, or list of characters, that indicate the start of a single-line comment.
- encoding – The name of the encoding used to decode the file content.
Examples: >>> from pypfilt.io import read_lookup_table, lookup >>> from pypfilt.time import Datetime >>> import datetime >>> path = "input_data.ssv" >>> with open(path, 'w') as f: ... _ = f.write('date value1 value2 value3\n') ... _ = f.write('2020-01-01 1.0 1.5 2.0\n') >>> time = Datetime() >>> table = read_lookup_table(path, time) >>> isinstance(table['date'][0], datetime.datetime) True >>> when = datetime.datetime(2020, 1, 1) >>> values = lookup(table, when) >>> len(values.shape) == 1 True >>> all(isinstance(value, float) for value in values) True
-
pypfilt.io.
lookup_values_count
(lookup_table)¶ Return the number of value columns in a lookup table.
-
pypfilt.io.
lookup
(lookup_table, when)¶ Return the values associated with a specific time.