Pipeline#
- class lumen.pipeline.Pipeline(*, source, table, schema=None, filters=None, **params)#
Pipeline encapsulates filters and transformations applied to a
lumen.sources.base.Sourcetable.A Pipeline ingests data from a
lumen.sources.base.Sourcetable or another Pipeline applying the declaredlumen.filters.base.Filter,lumen.transforms.base.Transformandlumen.transforms.sql.SQLTransformdefinitions. It can be used to drive one or more visual outputs or leveraged as a standalone component to encapsulate multiple data processing steps.
Parameters#
type: bool
default: True
Whether changes in filters, transforms and references automaticallytrigger updates in the data or whether an update has to be triggeredmanually using the update event or the update button in the UI.
type: pandas.DataFrame
default: None
The current data on this source.
type: list[lumen.Filter]
default: []
A list of Filters to apply to the source data.
type: str
default: 'Pipeline'
String identifier for this object.
type: lumen.Pipeline
default: None
Optionally a pipeline may be chained to another pipeline.
type: dict
default: None
The schema of the input data.
type: lumen.Source
default: None
The Source this pipeline is fed by.
type: list[lumen.transforms.SQLTransform]
default: []
A list of SQLTransforms to apply to the source data.
type: str
default: ''
The name of the table driving this pipeline.
type: list[lumen.Transform]
default: []
A list of Transforms to apply to the source data.
type: bool
default: False
Update event trigger (if manual update is set).
Methods#
- Pipeline.add_filter(filt: Filter | Type[Filter] | Widget, field: str | None = None, **kwargs)#
Add a filter to the pipeline.
- Pipeline.add_transform(transform: Transform, **kwargs)#
Add a (SQL)Transform to the pipeline.
- Parameters:
filt (Transform) – The Transform instance to add.
- Pipeline.chain(filters: List[Filter] | None = None, transforms: List[Transform] | None = None, sql_transforms: List[Transform] | None = None, **kwargs)#
Chains additional filtering, transform and sql_transform operations on an existing pipeline. Note that if one or more sql_transforms are provided the pipeline is cloned rather than applying the operations on top of the existing pipeline.
- Parameters:
- Return type:
- Pipeline.clone(**params) Pipeline#
Create a new instance of the pipeline with optionally overridden parameter values.
- Pipeline.get_schema()#
Generates a JSON schema for the current data held by the Pipeline.
- Returns:
schema – JSON schema for each column in the current data.
- Return type:
dict[str, any]
- Pipeline.precache(queries: Dict[str, Dict[str, List[Any]]] | List[Dict[str, Dict[str, Any]]]) None#
Populates the cache of the
lumen.sources.base.Sourcewith the provided queries.Queries can be provided in two formats:
A dictionary containing ‘filters’ and ‘variables’ dictionaries each containing lists of values to compute a cross-product for, e.g.
- {
- ‘filters’: {
<filter>’: [‘a’, ‘b’, ‘c’, …], …
}, ‘variables’: {
<variable>: [0, 2, 4, …], …
}
}
A list containing dictionaries of explicit values for each filter and variables.
- [{
‘filters’: {<filter>: ‘a’}, ‘variables’: {<variable>: 0}
}, {
‘filters’: {<filter>: ‘a’}, ‘variables’: {<variable>: 1}
]
- Pipeline.servable(title: str | None = None, location: bool | 'Location' = True, area: str = 'main', target: str | None = None) Viewable#
Serves the object or adds it to the configured pn.state.template if in a panel serve context, writes to the DOM if in a pyodide context and returns the Panel object to allow it to display itself in a notebook context.
- Parameters:
title (str) – A string title to give the Document (if served as an app)
location (boolean or panel.io.location.Location) – Whether to create a Location component to observe and set the URL location.
area (str (deprecated)) – The area of a template to add the component too. Only has an effect if pn.config.template has been set.
target (str) – Target area to write to. If a template has been configured on pn.config.template this refers to the target area in the template while in pyodide this refers to the ID of the DOM node to write to.
- Return type:
The Panel object itself
- Pipeline.show(title: str | None = None, port: int = 0, address: str | None = None, websocket_origin: str | None = None, threaded: bool = False, verbose: bool = True, open: bool = True, location: bool | 'Location' = True, **kwargs) threading.Thread | 'Server'#
Starts a Bokeh server and displays the Viewable in a new tab.
- Parameters:
title (str | None) – A string title to give the Document (if served as an app)
port (int (optional, default=0)) – Allows specifying a specific port
address (str) – The address the server should listen on for HTTP requests.
websocket_origin (str or list(str) (optional)) – A list of hosts that can connect to the websocket. This is typically required when embedding a server app in an external web site. If None, “localhost” is used.
threaded (boolean (optional, default=False)) – Whether to launch the Server on a separate thread, allowing interactive use.
verbose (boolean (optional, default=True)) – Whether to print the address and port
open (boolean (optional, default=True)) – Whether to open the server in a new browser tab
location (boolean or panel.io.location.Location) – Whether to create a Location component to observe and set the URL location.
- Returns:
server – Returns the Bokeh server instance or the thread the server was launched on (if threaded=True)
- Return type:
bokeh.server.Server or panel.io.server.StoppableThread
- Pipeline.to_spec(context: Dict[str, Any] | None = None) Dict[str, Any]#
Exports the full specification to reconstruct this component.
- Parameters:
context (Dict[str, Any]) – Context contains the specification of all previously serialized components, e.g. to allow resolving of references.
- Return type:
Declarative specification of this component.