API reference¶

FOREST is a command line application built on top of the Bokeh library that harnesses the power of scientific software to visualise meteorological and oceanographic forecasts alongside observations.

Redux design pattern¶

As applications grow the number of components vying to keep their state synchronised becomes unwieldy. A better way to manage state is centralise it so that there is a single source of truth. The consequence of which is easy to replay, serialise and rehydrate applications.

For further reading, check out the redux.js docs

https://redux.js.org/introduction/motivation

It also enables an easily unit testable system since state updates are pure function with easy to separate single responsibility components. Middleware components are plug and play, drop one in to add behaviour to your app without fear of altering existing behaviour.

class forest.redux.Store(reducer, initial_state=None, middlewares=())[source]¶

Observable state container

The redux design pattern is a simple way to keep track of state changes. A reducer combines an action with the current state to produce the next state. The reducer should be a pure function in the sense of not having side effects.

Non-pure behaviour can be incorporated through the use of middleware. Middleware takes an action and either passes it on, filters it, enriches it or emits new actions.

The store is an observable that emits states, views can register themselves with the store to receive the latest states as and when they are created.

>>> store.add_subscriber(listener)

Parameters:	reducer – function combines action and state to produce new state initial_state – optional initial state, default {} middlewares – list of middleware functions that intercept actions

static bind(middleware, store, actions)[source]¶: Flat map action generators from middleware into action generator

dispatch(action)[source]¶

Apply reducer and notify listeners of new state

Parameters:	action – plain dict consumed by the reducer

static pure(action)[source]¶: Embed action into action generator

sync_process(action)[source]¶: Pass action through middleware/reducer pipeline

forest.redux.combine_reducers(*reducers)[source]¶

Simple combine passes action and state to all reducers

Returns:	reducer function

Application state¶

Although the state in a redux store is defined by reducer functions, engineers need documentation to extend and understand State. Python dataclasses are a natural fit to make state self-describing.

Expressing data structure as a nested hierarchy of types allows readers of the code to understand how state is organised. It also allows for type-checking to simplify functions that manipulate state.

State can be generated programmatically, converted to/from dict to be compatible with reducers and to make it easier to serialize.

>>> state = forest.state.State()
>>> state.colorbar.name
'Viridis'

Converting to/from dict can be achieved using State.to_dict() and State.from_dict()

>>> s1 = forest.state.State()
>>> d = s1.to_dict()
>>> type(d)
<class 'dict'>
>>> s2 = forest.state.State.from_dict(d)
>>> s1 == s2
True

The only caveat to be aware of while mapping to/from dict is that State implements default values for missing entries. A default State is not equal to an empty dict.

>>> forest.state.State().to_dict() == {}
False

Note

State structure may change in future releases, backwards compatibility is not guaranteed

class forest.state.Bokeh(html_loaded: bool = False)[source]¶

Additional bokeh state

Image streaming behaves inconsistently when data is streamed before the DOM is ready, image_shape[t] is undefined errors are triggered in the compiled JS

Note

HTML loaded is merely convenience and may be unnecessary in future bokeh releases

class forest.state.Borders(line_color: str = 'black', visible: bool = False)[source]¶

Cartopy border overlay settings

Parameters:	line_color (str) – Color of coastlines and country borders visible (bool) – Turn all lines on/off

class forest.state.Colorbar(name: str = 'Viridis', names: list = <factory>, number: int = 256, numbers: list = <factory>, limits: forest.state.ColorbarLimits = <factory>, low: float = 0.0, high: float = 1.0, reverse: bool = False, invisible_min: bool = False, invisible_max: bool = False)[source]¶

Colorbar settings allow users to change palettes and limits based on data or user-specified limits

Parameters:	name – bokeh palette name number – bokeh palette number limits (ColorbarLimits) – user and column_data_source limits reverse – reverse color palette order invisible_min – hide/show values below minimum invisible_max – hide/show values above maximum

class forest.state.ColorbarLimits(origin: str = 'column_data_source', column_data_source: forest.state.Limits = <factory>, user: forest.state.Limits = <factory>)[source]¶

Define user and column data source limits

Parameters:	origin (str) – either ‘user’ or ‘column_data_source’ column_data_source (Limits) – column_data_source limits user (Limits) – user limits

class forest.state.LayerMode(state: str = 'add', index: int = 0)[source]¶

Data to control UI presented to user

Contains meta-data to indicate whether a layer is being edited or added. If the layer is being edited an index can be used to specify settings to overwrite.

Parameters:	state – Edit mode, either ‘edit’ or ‘add’ index – Index of layer being edited

class forest.state.Layers(figures: int = 1, index: dict = <factory>, active: list = <factory>, mode: forest.state.LayerMode = <factory>)[source]¶

Layer settings

Parameters:	figures – Number of figures to display index – Map layer index to settings active – List of active layers mode (LayerMode) – Edit/new mode to define UI

class forest.state.Limits(low: float = 0.0, high: float = 1.0)[source]¶

Color map extent, high and low represent upper and lower limits respectively

Parameters:	low (float) – lower limit high (float) – upper limit

class forest.state.Position(x: float = 0.0, y: float = -1000000000.0)[source]¶

X/Y position in WebMercator coordinates related to user interaction

Parameters:	x – coordinate of tap event y – coordinate of tap event

class forest.state.Presets(active: int = 0, labels: dict = <factory>, meta: dict = <factory>)[source]¶

Re-usable layer settings

Presets are cooked up once and re-used anywhere, they can also be tweaked on the fly and instantly made available to all layers using them. They can also be serialised to disk to store/re-load them as needed

Parameters:	active (dict) – currently chosen preset labels – map index to label meta – data used by user interface

class forest.state.State(pattern: str = None, patterns: list = <factory>, variable: str = None, variables: list = <factory>, initial_time: datetime.datetime = datetime.datetime(1970, 1, 1, 0, 0), initial_times: list = <factory>, valid_time: datetime.datetime = datetime.datetime(1970, 1, 1, 0, 0), valid_times: list = <factory>, pressure: float = 0.0, pressures: list = <factory>, colorbar: forest.state.Colorbar = <factory>, layers: forest.state.Layers = <factory>, dimension: dict = <factory>, tile: forest.state.Tile = <factory>, tools: forest.state.Tools = <factory>, position: forest.state.Position = <factory>, presets: forest.state.Presets = <factory>, borders: forest.state.Borders = <factory>, bokeh: forest.state.Bokeh = <factory>)[source]¶

Application State container

Nested data structure to define components, views and behaviour.

Parameters:

pattern – User-selected value
patterns – Dataset-specific values
variable – User-selected value
variables – Dataset-specific values
initial_time – User-selected value
initial_times – Dataset-specific values
valid_time – User-selected value
valid_times – Dataset-specific values
pressure – User-selected value
pressures – Dataset-specific values
layers (Layers) – Layer-specific settings
colorbar (Colorbar) – Color mapper controls
tile (Tile) – Web map tiling configuration
tools (Tools) – Turn profile/time_series on/off
position (Position) – Used by tools to determine geographic position
presets (Presets) – Save colorbar settings for later re-use
borders (Borders) – Cartopy coastline, lakes and border settings
bokeh (Bokeh) – Additional bokeh state

classmethod from_dict(data: dict)[source]¶

Factory method to convert from dict to State

Returns:	State instance
Return type:	State

to_dict()[source]¶

Map to dict representation of State

Returns:	dictionary containing nested state data
Return type:	dict

class forest.state.Tile(name: str = 'Open street map', labels: bool = False)[source]¶

Web map tiling user-settings

Parameters:	name (str) – Keyword to specify WMTS source labels (bool) – Turn overlay labels on/off

class forest.state.Tools(time_series: bool = False, profile: bool = False)[source]¶

Flags to specify active tools

Parameters:	time_series (bool) – Turn time series widget on/off profile – Turn profile widget on/off

Rx - Functional reactive programming¶

The basic data structue in functional reactive programming is a Stream

class forest.rx.Stream[source]¶

Sequence of events

An event is a value passed to a callback or listener. Most data structures exist in space, e.g. in RAM or on disk, but Streams exist in time.

Common operations on streams include map() and filter().

distinct(comparator=None)[source]¶

Remove repeated items

Parameters:	comparator – f(x, y) that returns True if x == y
Returns:	new stream with duplicate events removed

filter(f)[source]¶

Emit items that pass a predicate test

Parameters:	f – predicate function True keeps value False discards value

listen_to(observable)[source]¶

Re-transmit events on another observable

Returns:	current stream

map(f)[source]¶

Make new stream by applying f to values

Returns:	new stream that emits f(x)

Observer pattern¶

The most basic observer pattern simply calls one at a time each subscriber with a value that represents information important to the subscriber.

class forest.observe.Observable[source]¶

Basic observer design pattern

add_subscriber(method)[source]¶: Append method to list of subscribers

notify(value)[source]¶: Call subscribers with value

Time series¶

Support for time series uses the redux design pattern. The View reacts to State changes.

class forest.series.SeriesView(figure, loaders)[source]¶

Time series view

Responsible for keeping the lines on the series figure up to date.

classmethod from_groups(figure, groups)[source]¶: Factory method to load from FileGroup objects

render(initial_time, variable, x, y, visible, pressure=None)[source]¶: Update data for a particular application setting

class forest.series.SeriesLoader(paths)[source]¶: Time series loader

class forest.series.SeriesLocator(paths)[source]¶: Helper to find files related to Series

Selector¶

The full application state can be unwieldy for individual views to parse, especially if the details of its internals change. Instead of relying on Views to translate state into values, selectors can do that job on behalf of the view.

forest.series.select_args(state)[source]¶

Select args needed by SeriesView.render()

Note

If all criteria are not present None is returned

Returns:	args tuple or None

Color palette¶

Helpers to choose color palette(s), limits etc.

UI components¶

The following components wire up the various bokeh widgets and event handlers to actions and react to changes in state. They are typically used in the following manner.

>>> component = Component().connect(store)
>>> bokeh.layouts.column(component.layout)

class forest.colors.ColorPalette[source]¶

Color palette user interface

connect(store)[source]¶: Connect component to Store

on_name(event)[source]¶: Event-handler when a palette name is selected

on_number(event)[source]¶: Event-handler when a palette number is selected

on_reverse(attr, old, new)[source]¶: Event-handler when reverse toggle is changed

props()[source]¶: Helper to get widget settings

render(props)[source]¶: Render component from properties derived from state

class forest.colors.UserLimits[source]¶

User controlled color mapper limits

connect(store)[source]¶

Connect component to Store

Convert state stream to properties used by render method.

Parameters:	store (`forest.redux.Store`) – instance to dispatch actions and listen to state changes

on_input_high(attr, old, new)[source]¶: Event-handler to set user high

on_input_low(attr, old, new)[source]¶: Event-handler to set user low

on_invisible_max(attr, old, new)[source]¶: Event-handler when invisible_max toggle is changed

on_invisible_min(attr, old, new)[source]¶: Event-handler when invisible_min toggle is changed

props()[source]¶: Helper to get current state of widgets

render(props)[source]¶: Update user-defined limits inputs

class forest.colors.SourceLimits[source]¶

Event stream listening to collection of ColumnDataSources

Translates column data source on_change events into domain specific actions, e.g. set_source_limits(). Instead of connecting to a forest.redux.Store, simply subscribe store.dispatch to action events.

>>> source_limits = SourceLimits()
>>> for source in sources:
...     source_limits.add_source(source)
>>> source_limits.connect(store)

Note

Unlike a typical component there is no layout property to attach to a bokeh document

add_source(source)[source]¶: Add ColumnDataSource to listened sources

connect(store)[source]¶: Connect events to the Store

limits(sources)[source]¶: Calculate limits from underlying sources

on_change(attr, old, new)[source]¶: Generate action from bokeh event

remove_source(source)[source]¶: Remove ColumnDataSource from listened sources

Most components are not interested in the full application state. The connect() method and state_to_props() are provided to only notify UI components when relevant state updates.

forest.colors.connect(view, store)[source]¶

Connect component to Store

UI components connected to a Store only need to be notified when a change occurs that is relevant to them, all other state updates can be safely ignored.

To implement component specific updates this helper method listens to store dispatch events, converts them to a stream of states, maps the states to props and filters out duplicates.

forest.colors.state_to_props(state)[source]¶

Map state to props relevant to component

Parameters:	state – dict representing full application state
Returns:	`state["colorbar"]` or `None`

Reducer¶

A reducer combines the current state with an action to produce a new state

forest.colors.reducer(state, action)[source]¶

Reducer for colorbar actions

Combines current state with an action to produce the next state

Returns:	new state
Return type:	dict

Middleware¶

Middleware pre-processes actions prior to the reducer

forest.colors.palettes(store, action)[source]¶

Color palette middleware

Encapsulates colorbar user interface logic. For example, if a user has chosen to fix their data limits, then set_limit actions generated by column data source changes are ignored

Note

middleware is an action generator

Helpers¶

Convenient functions to simplify color bar settings

forest.colors.defaults()[source]¶

Default color palette settings

{
    "name": "Viridis",
    "names": palette_names(),
    "number": 256,
    "numbers": palette_numbers("Viridis"),
    "low": 0,
    "high": 1,
    "reverse": False,
    "invisible_min": False,
    "invisible_max": False,
}

Note

incomplete settings create unintuitive behaviour when restoring from a previously saved palette

Returns:	dict representing default colorbar

forest.colors.palette_names()[source]¶

All palette names

Returns:	list of valid bokeh palette names

forest.colors.palette_numbers(name)[source]¶

Helper to choose available color palette numbers

Returns:	list of valid bokeh palette numbers

Actions¶

Actions are small pieces of data used to communicate with other parts of the system. Reducers and middleware functions can interpret their contents and either update state or generate new actions

forest.colors.set_colorbar(options)[source]¶: Action to set multiple settings at once

forest.colors.set_reverse(flag)[source]¶: Action to reverse color palette colors

forest.colors.set_palette_name(name)[source]¶: Action to set color palette name

forest.colors.set_palette_names(names)[source]¶: Action to set all available palettes

forest.colors.set_palette_number(number)[source]¶: Action to set color palette size

forest.colors.set_palette_numbers(numbers)[source]¶: Action to set available levels for color palette

forest.colors.set_source_limits(low, high)[source]¶: Action to set colorbar limits from column data sources

forest.colors.set_user_high(high)[source]¶: Action to set user defined colorbar higher limit

forest.colors.set_user_low(low)[source]¶: Action to set user defined colorbar lower limit

forest.colors.set_invisible_min(flag)[source]¶: Action to mask out data below colour bar limits

forest.colors.set_invisible_max(flag)[source]¶: Action to mask out data below colour bar limits

Key press interaction¶

This module provides a KeyPress observable to enable server side Python functions to subscribe to key events.

In addtion there is also a middleware function, navigate(), that maps key actions to navigation actions, e.g. next_valid_time() etc.

class forest.keys.KeyPress[source]¶

Key press server-side observable

To add this to an existing document, add the hidden_button to the document and update templates/index.html to click the button on page load

>>> key_press = KeyPress()
>>> document.add_root(key_press.hidden_button)

To observe the stream of actions generated by user key press events simply register a function via the subscribe method

>>> key_press.subscribe(print)

Note

KeyPress.hidden_button must be added to the document to allow JS hack to initialise callbacks

forest.keys.navigate(store, action)[source]¶

Middleware to interpret key press events

It implements the following mapping of actions to allow other parts of the system to interpret the action

From	To
ArrowRight	Next valid time
ArrowLeft	Previous valid time
ArrowUp	Next initial time
ArrowDown	Previous initial time

Note

Non key press actions are passed on unaltered

Parameters:	store – `forest.redux.Store` instance action – incoming action

forest.keys.press(code)[source]¶

Key press action creator

Parameters:	code – str representing browser event.code
Returns:	dict representing action

Configure application¶

This module implements parsers and data structures needed to configure the application. It supports richer settings than those that can be easily represented on the command line by leveraging file formats such as YAML and JSON that are widely used to configure applications.

class forest.config.Config(data)[source]¶

Configuration data structure

This high-level object represents the application configuration. It is file format agnostic but has helper methods to initialise itself from disk or memory.

Note

This class is intended to provide the top-level configuration with low-level details implemented by specialist classes, e.g. FileGroup which contains meta-data for files

Parameters:	data – native Python data structure representing application settings

edition¶: Text format conventions

features¶: Dict of user-defined feature toggles

classmethod from_files(files, file_type='unified_model')[source]¶

Configure using list of file names and a file type

Parameters:	files – list of file names file_type – keyword to apply to all files
Returns:	instance of `Config`

classmethod load(path, variables=None)[source]¶

Parse settings from either YAML or JSON file on disk

The configuration can be controlled elegantly through a text file. Groups of files can be specified in a list.

Note

Relative or absolute directories are declared through the use of a leading /

files:
    - label: Trial
      pattern: "${TRIAL_DIR}/*.nc"
    - label: Control
      pattern: "${CONTROL_DIR}/*.nc"
    - label: RDT
      pattern: "${RDT_DIR}/*.json"
      file_type: rdt

Parameters:	path – JSON/YAML file to load variables – dict of key/value pairs used by `string.Template`
Returns:	instance of `Config`

classmethod loads(text, variables=None)[source]¶

Parse settings from either YAML or JSON string

The configuration can be controlled elegantly through a text file. Groups of files can be specified in a list.

Note

Relative or absolute directories are declared through the use of a leading /

files:
    - label: Trial
      pattern: "${TRIAL_DIR}/*.nc"
    - label: Control
      pattern: "${CONTROL_DIR}/*.nc"
    - label: RDT
      pattern: "${RDT_DIR}/*.json"
      file_type: rdt

Parameters:	text – JSON/YAML string variables – dict of key/value pairs used by `string.Template`
Returns:	instance of `Config`

presets_file¶

Colorbar presets JSON file

A location on disk where colorbar settings can be saved/loaded. If the file does not exist it will be created by the application.

Use the following syntax to declare the presets file location

presets:
  file: ${HOME}/example/preset-save.json

Returns:	location on disk to save colorbar presets

use_web_map_tiles¶

Turns web map tiling backgrounds on/off

use_web_map_tiles: false

Note

This is best used during development if an internet connection is not available

class forest.config.FileGroup(label, pattern, locator='file_system', file_type='unified_model', directory=None, database_path=None)[source]¶

Meta-data needed to describe group of files

To describe a collection of related files extra meta-data is needed. For example, the type of data contained within the files or how data is catalogued and searched.

Note

This class violates the integration separation principle (ISP) all driver settings are included in the same constructor

Parameters:	label – decription used by buttons and tooltips pattern – wildcard pattern used by either SQL or glob locator – keyword describing search method (default: ‘file_system’) file_type – keyword describing file contents (default: ‘unified_model’) directory – leaf/absolute directory where file(s) are stored (default: None)

forest.config.load_config(path)[source]¶: Load configuration from a file

forest.config.from_files(files, file_type)[source]¶: Define configuration with a list of files

Geographic utilities module¶

Module to handle projection and sampling iof points for imaging.

forest.geo.stretch_image(lons, lats, values, plot_height=None, plot_width=None)[source]¶

Do the mapping from image data to the format required by bokeh for plotting.

Parameters:	lons – Numpy array with latitude values for the image data. lats – Numpy array with longitude values for the image data. values – Numpy array of image data, with dimensions matching the size of latitude and longitude arrays.
Returns:	A dictionary that can be used with the bokeh image glyph.

forest.geo.web_mercator(lons, lats)[source]¶

forest.geo.plate_carree(x, y)[source]¶

Presets¶

User configured settings can be saved, edited and deleted.

UI components¶

class forest.presets.PresetUI[source]¶

User interface to load/save/edit presets

>>> preset_ui = PresetUI().connect(store)

connect(store)[source]¶: Convenient method to map state to props needed by render

on_load(attr, old, new)[source]¶: Notify listeners that a load action has taken place

on_save()[source]¶: Notify listeners that a save action has taken place

Reducer¶

forest.presets.reducer(state, action)[source]¶

Presets reducer

Returns:	next state

Middleware¶

Middleware pre-processes actions prior to the reducer

forest.presets.middleware(store, action)[source]¶

Presets middleware

Generates actions given current state and an incoming action. Encapsulates the business logic surrounding saving, editing and creating presets.

Helpers¶

forest.presets.state_to_props(state)[source]¶: Converts application state to props used by user interface

Actions¶

A simple grammar used to communicate between components.

forest.presets.save_preset(label)[source]¶: Action to save a preset

forest.presets.load_preset(label)[source]¶: Action to load a preset by label

forest.presets.remove_preset()[source]¶: Action to remove a preset

forest.presets.set_default_mode()[source]¶: Action to select default display mode

forest.presets.set_edit_mode()[source]¶: Action to select edit display mode

forest.presets.set_edit_label(label)[source]¶: Action to set edit mode label

forest.presets.on_save(label)[source]¶: Action to signal save clicked

forest.presets.on_edit()[source]¶: Action to signal edit clicked

forest.presets.on_new()[source]¶: Action to signal new clicked

forest.presets.on_cancel()[source]¶: Action to signal cancel clicked

Services¶

Application wide services to de-couple components. For example, components need access to navigation to populate dropdown menus and widgets.

Navigation¶

Datasets dimensions are often represented by large arrays, instead of storing the arrays in the forest.redux.Store a service is provided to load the arrays from key pieces of information. The key information is stored in the application state to allow reproduction at a later date.

class forest.services.NavigatorServiceLocator[source]¶

De-couples client-code from Navigator construction

Note

This service locator is accessed as forest.services.navigation module level constant

Configured at run-time by calling add_dataset(key, dataset)

>>> forest.services.navigation.add_dataset("name", dataset)

And used by views and components as follows

>>> navigator = forest.services.navigation.get_navigator("name")

add_dataset(name, dataset)[source]¶

Register Dataset(s) with service at run-time

Parameters:	name – key used to locate navigator service dataset – `forest.drivers.Dataset` instance

get_navigator(dataset_name)[source]¶

Find appropriate Navigator

Note

If service unavailable a NullNavigator is returned to prevent system crash

Parameters:	name – key used to locate navigator service
Returns:	`forest.drivers.Dataset` instance

class forest.services.NullNavigator[source]¶

Empty container to allow client-code to work if service not found

initial_times(pattern, variable)[source]¶

Returns:	empty list

pressures(pattern, variable, initial_time)[source]¶

Returns:	empty list

valid_times(pattern, variable, initial_time)[source]¶

Returns:	empty list

variables(pattern)[source]¶

Returns:	empty list