IntakeSQLSource type: intake_sql#
- class lumen.sources.intake_sql.IntakeSQLSource(*, filter_in_sql, catalog, dask, uri, load_schema, cache_dir, cache_per_query, cache_with_dask, root, shared, name)#
IntakeSQLSource extends the IntakeSource with support for SQL data.
In addition to the standard intake support for reading catalogs the IntakeSQLSource computes the schema by querying the database instead of loading all the data into memory and allows for SQLTransform to be applied when querying the SQL database.
Parameters#
type: dict
default: None
An inlined Catalog specification.
type: bool
default: False
Whether to return a dask DataFrame.
type: bool
default: True
type: bool
default: True
Whether to load the schema
type: str
default: ''
URI of the catalog file.
Methods#
- IntakeSQLSource.clear_cache(*events: Event)#
Clears any cached data.
- IntakeSQLSource.get(table, **query)#
Applies SQL Transforms, creating new temp catalog on the fly and querying the database.
- IntakeSQLSource.get_schema(table: str | None = None, limit: int | None = None) Dict[str, Dict[str, Any]] | Dict[str, Any]#
Returns JSON schema describing the tables returned by the Source.
- Parameters:
table (str | None) – The name of the table to return the schema for. If None returns schema for all available tables.
limit (int | None) – Limits the number of rows considered for the schema calculation
- Returns:
JSON schema(s) for one or all the tables.
- Return type:
dict
- IntakeSQLSource.get_sql_expr(table)#
- IntakeSQLSource.get_tables()#
Returns the list of tables available on this source.
- Returns:
The list of available tables on this source.
- Return type:
list
- IntakeSQLSource.to_spec(context: Dict[str, Any] | None = None) Dict[str, Any]#
Exports the full specification to reconstruct this component.
- Return type:
Resolved and instantiated Component object