Analyses¶
Base class for deterministic analyses run by AnalysisAgent.
lumen.ai.analysis
¶
TContext = Mapping[str, Any]
module-attribute
¶
Analysis
¶
Bases:
The purpose of an Analysis is to perform custom actions tailored to a particular use case and/or domain. It is a ParameterizedFunction which implements an applies method to confirm whether the analysis applies to a particular dataset and a call method which is given a pipeline and must return another valid component.
agents = param.List(default=[], doc='List of available agents that can be invoked by the analysis.')
class-attribute
instance-attribute
¶
autorun = param.Boolean(default=True, doc='Whether to automatically run the analysis.')
class-attribute
instance-attribute
¶
columns = param.List(default=[], constant=True, doc='\n The columns required for the analysis. May use tuples to declare that one of\n the columns must be present.')
class-attribute
instance-attribute
¶
interface = param.ClassSelector(class_=ChatFeed, doc='\n The ChatInterface instance that will be used to stream messages.')
class-attribute
instance-attribute
¶
message = param.String(default='', doc='The message to display on interface when the analysis is run.')
class-attribute
instance-attribute
¶
applies(pipeline)
async
classmethod
¶
controls(context)
¶
Component
¶
Bases:
Baseclass for all Lumen component types including Source, Filter,
Transform, Variable and View types. Components must implement
serialization and deserialization into a specification dictionary
via the from_spec and to_spec protocol. Additonally they
should implement validation.
refs
property
¶
from_spec(spec)
classmethod
¶
Creates a Component instance from a specification.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
spec
|
|
Specification declared as a dictionary of parameter values or a string referencing a source in the sources dictionary. |
required |
Returns:
| Type | Description |
|---|---|
Resolved and instantiated Component object
|
|
to_spec(context=None)
¶
Exports the full specification to reconstruct this component.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
context
|
|
Context contains the specification of all previously serialized components, e.g. to allow resolving of references. |
None
|
Returns:
| Type | Description |
|---|---|
|
Declarative specification of this component. |
validate(spec, context=None)
classmethod
¶
Validates the component specification given the validation context.
Arguments
spec: dict | str The specification for the component being validated (or a referene to the component) context: dict Validation context contains the specification of all previously validated components, e.g. to allow resolving of references.
Returns:
| Type | Description |
|---|---|
Validated specification.
|
|
Pipeline
¶
Bases:
Pipeline encapsulates filters and transformations applied to a
:class:lumen.sources.base.Source table.
A Pipeline ingests data from a
:class:lumen.sources.base.Source table or another Pipeline
applying the declared :class:lumen.filters.base.Filter,
:class:lumen.transforms.base.Transform and
:class:lumen.transforms.sql.SQLTransform definitions. It can be
used to drive one or more visual outputs or leveraged as a
standalone component to encapsulate multiple data processing
steps.
auto_update = param.Boolean(default=True, constant=True, doc='\n Whether changes in filters, transforms and references automatically\n trigger updates in the data or whether an update has to be triggered\n manually using the update event or the update button in the UI.')
class-attribute
instance-attribute
¶
control_panel
property
¶
data = DataFrame(doc='The current data on this source.')
class-attribute
instance-attribute
¶
dataframe_backend = param.Selector(default=None, objects=[None, *DATAFRAME_BACKENDS], doc='\n The dataframe library `Pipeline.data` is returned as. Leave unset to\n get whatever the Source produced, which avoids a conversion.')
class-attribute
instance-attribute
¶
filters = param.List(item_type=Filter, doc='\n A list of Filters to apply to the source data.')
class-attribute
instance-attribute
¶
pipeline = param.ClassSelector(class_=None, doc='\n Optionally a pipeline may be chained to another pipeline.')
class-attribute
instance-attribute
¶
refs
property
¶
schema = param.Dict(doc='The schema of the input data.')
class-attribute
instance-attribute
¶
source = param.ClassSelector(class_=Source, doc='\n The Source this pipeline is fed by.')
class-attribute
instance-attribute
¶
sql_pushdown = param.Boolean(default=True, doc='\n Whether to narrow the query to the columns the transforms actually\n need when the Source supports SQL, instead of fetching every column\n and projecting in pandas. Only applied where the result is provably\n identical; set to False to always fetch the full table.')
class-attribute
instance-attribute
¶
sql_transforms = param.List(item_type=SQLTransform, doc='\n A list of SQLTransforms to apply to the source data.')
class-attribute
instance-attribute
¶
table = param.String(doc='\n The name of the table driving this pipeline.')
class-attribute
instance-attribute
¶
transforms = param.List(item_type=Transform, doc='\n A list of Transforms to apply to the source data.')
class-attribute
instance-attribute
¶
update = param.Event(label='Apply update', doc='\n Update event trigger (if manual update is set).')
class-attribute
instance-attribute
¶
add_filter(filt, field=None, **kwargs)
¶
Add a filter to the pipeline.
Arguments
filt: Filter | Type[Filter] The filter instance or filter type to add. field: str | None The field to filter on (required to instantiate Filter type).
add_transform(transform, **kwargs)
¶
Add a (SQL)Transform to the pipeline.
Arguments
filt: Transform The Transform instance to add.
chain(filters=None, transforms=None, sql_transforms=None, **kwargs)
¶
Chains additional filtering, transform and sql_transform operations on an existing pipeline. Note that if one or more sql_transforms are provided the existing table will be mirrored into a DuckDB database.
Arguments
filters: List[Filter] | None Additional filters to apply on top of existing pipeline. transforms: List[Transform] | None Additional transforms to apply on top of existing pipeline. sql_transforms: List[SQLTransform] | None Additional filters to apply on top of existing pipeline.
Returns:
| Type | Description |
|---|---|
|
|
clone(**params)
¶
Create a new instance of the pipeline with optionally overridden parameter values.
from_spec(spec, source=None, source_filters=None)
classmethod
¶
Creates a Pipeline from a specification.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
spec
|
|
Specification declared as a dictionary of parameter values or a string referencing a source in the sources dictionary. |
required |
Returns:
| Type | Description |
|---|---|
Resolved and instantiated Pipeline object
|
|
get_dataset()
¶
Return the pipeline data as a gridded xarray.Dataset when the
source can produce one (xarray-backed, via to_dataset) and this is a
plain source query -- no chained pipeline, ParamFilter, or Python
transforms. Returns None otherwise so callers fall back to the
long-form frame.
Lets gridded views render from a compact xarray grid instead of pivoting the long-form data back to xarray.
get_schema()
¶
Generates a JSON schema for the current data held by the Pipeline.
Returns:
| Name | Type | Description |
|---|---|---|
schema |
|
JSON schema for each column in the current data. |
precache(queries)
¶
Populates the cache of the :class:lumen.sources.base.Source with the provided queries.
Queries can be provided in two formats:
-
A dictionary containing 'filters' and 'variables' dictionaries each containing lists of values to compute a cross-product for, e.g.
{ 'filters': {
': ['a', 'b', 'c', ...], ... }, 'variables': { : [0, 2, 4, ...], ... } } - A list containing dictionaries of explicit values for each filter and variables. [{ 'filters': {
: 'a'}, 'variables': { : 0} }, { 'filters': { : 'a'}, 'variables': { : 1} }, ... ]
to_spec(context=None)
¶
Exports the full specification to reconstruct this component.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
context
|
|
Context contains the specification of all previously serialized components, e.g. to allow resolving of references. |
None
|
Returns:
| Type | Description |
|---|---|
Declarative specification of this component.
|
|
traverse(type)
¶
Returns all Filter or Transform objects in a potentially chained pipeline.
validate(spec, context=None)
classmethod
¶
get_data(pipeline)
async
¶
Return pipeline.data as pandas, off the main thread.
result_to_dataframe, the LLM code sandbox and the plotting agents are all written against pandas.