WebsiteSource type: live#
- class lumen.sources.base.WebsiteSource(*, urls, cache_dir, cache_per_query, cache_with_dask, root, shared, name)#
WebsiteSource queries whether a website responds with a 400 status code.
Parameters#
type: list[Any]
default: []
URLs of the websites to monitor.
Methods#
- WebsiteSource.clear_cache(*events: Event)#
Clears any cached data.
- WebsiteSource.get(table: str, **query) DataFrame#
Return a table; optionally filtered by the given query.
- Parameters:
table (str) – The name of the table to query
query (dict) – A dictionary containing all the query parameters
- Returns:
A DataFrame containing the queried table.
- Return type:
DataFrame
- WebsiteSource.get_schema(table: str | None = None, limit: int | None = None) Dict[str, Dict[str, Any]] | Dict[str, Any]#
Returns JSON schema describing the tables returned by the Source.
- Parameters:
table (str | None) – The name of the table to return the schema for. If None returns schema for all available tables.
limit (int | None) – Limits the number of rows considered for the schema calculation
- Returns:
JSON schema(s) for one or all the tables.
- Return type:
dict
- WebsiteSource.get_tables() List[str]#
Returns the list of tables available on this source.
- Returns:
The list of available tables on this source.
- Return type:
list
- WebsiteSource.to_spec(context: Dict[str, Any] | None = None) Dict[str, Any]#
Exports the full specification to reconstruct this component.
- Return type:
Resolved and instantiated Component object