Sources

The Source classes are the data provider for the entire dashboard. They handle querying some remote source such as a REST server, a website or some other provider and provide methods to return a set of tables (using the .get method) and JSON schemas describing the tables (using the .get_schema method).

class lumen.sources.Source(**params)

A Source provides a set of tables which declare their available fields. The Source must also be able to return a schema describing the types of the variables and indexes in each table and allow querying the data.

cache_dir = param.String(readonly=False)

Whether to enable local cache and write file to disk.

shared = param.Boolean(readonly=False)

Whether the Source can be shared across all instances of the dashboard. If set to True the Source will be loaded on initial server load.

clear_cache()

Clears any cached data.

classmethod from_spec(spec, sources={}, root=None)

Creates a Source object from a specification. If a Source specification references other sources these may be supplied in the sources dictionary and be referenced by name.

Parameters
  • spec (dict or str) – Specification declared as a dictionary of parameter values or a string referencing a source in the sources dictionary.

  • sources (dict) – Dictionary of other Source objects

  • root (str) – Root directory where dashboard specification was loaded from.

Returns

Return type

Resolved and instantiated Source object

get(table, **query)

Return a table; optionally filtered by the given query.

Parameters
  • table (str) – The name of the table to query

  • query (dict) – A dictionary containing all the query parameters

Returns

A DataFrame containing the queried table.

Return type

DataFrame

get_schema(table=None)

Returns JSON schema describing the tables returned by the Source.

Parameters

table (str or None) – The name of the table to return the schema for. If None returns schema for all available tables.

Returns

JSON schema(s) for one or all the tables.

Return type

dict

get_tables()

Returns the list of tables available on this source.

Returns

The list of available tables on this source.

Return type

list

property panel

A Source can return a Panel object which displays information about the Source or controls how the Source queries data.

Source types

class lumen.sources.JoinedSource(**params)

A JoinedSource applies a join on two or more sources returning new table(s) with data from all sources. It iterates over the tables specification and merges the specified tables from the declared sources on the supplied index.

In this way multiple tables from multiple sources can be merged. Individual tables from sources that should not be joined may also be surfaced by declaring a single source and table in the specification.

As a simple example we may have sources A and B, which contain tables ‘foo’ and ‘bar’ respectively. We now want to merge these tables on column ‘a’ in Table A with column ‘b’ in Table B:

{‘new_table’: [

{‘source’: ‘A’, ‘table’: ‘foo’, ‘index’: ‘a’}, {‘source’: ‘B’, ‘table’: ‘bar’, ‘index’: ‘b’}

]}

The joined source will now publish the “new_table” with all columns from tables “foo” and “bar” except for the index column from table “bar”, which was merged with the index column “a” from table “foo”.

Parameters inherited from:

lumen.sources.base.Source: cache_dir, shared

sources = param.Dict(readonly=False)

A dictionary of sources indexed by their assigned name.

tables = param.Dict(readonly=False)

A dictionary with the names of the joined sources as keys and a specification of the source, table and index to merge on. {“new_table”: [ {‘source’: <source_name>, ‘table’: <table_name>, ‘index’: <index_name> }, {‘source’: <source_name>, ‘table’: <table_name>, ‘index’: <index_name> }, … ]}

class lumen.sources.FileSource(**params)

Loads CSV, Excel, JSON and Parquet files using pandas.read_* or dask.read_* functions.

Parameters inherited from:

lumen.sources.base.Source: cache_dir, shared

dask = param.Boolean(readonly=False)

Whether to return a Dask dataframe.

kwargs = param.Dict(readonly=False)

Keyword arguments to the pandas/dask loading function.

tables = param.ClassSelector(readonly=False)

List or dictionary of tables to load. If a list is supplied the names are computed from the filenames, otherwise the keys are the names. The values must filepaths or URLs to the data: { ‘local’ : ‘/home/user/local_file.csv’, ‘remote’: ‘https://test.com/test.csv’ } if the filepath does not have a declared extension an extension may be provided in a list or tuple, e.g.: {‘table’: [‘http://test.com/api’, ‘json’]}

use_dask = param.Boolean(readonly=False)

Whether to use dask to load files.

class lumen.sources.PanelSessionSource(**params)

Parameters inherited from:

lumen.sources.base.Source: cache_dir, shared

endpoint = param.String(readonly=False)

urls = param.List(readonly=False)

URL of the websites to monitor.

timeout = param.Parameter(readonly=False)

class lumen.sources.RESTSource(**params)

Queries a REST API which is expected to conform to the monitoring REST API specification.

Parameters inherited from:

lumen.sources.base.Source: cache_dir, shared

url = param.String(readonly=False)

URL of the REST endpoint to monitor.

class lumen.sources.WebsiteSource(**params)

Queries whether a website responds with a 400 status code.

Parameters inherited from:

lumen.sources.base.Source: cache_dir, shared

urls = param.List(readonly=False)

URLs of the websites to monitor.